Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TUTORIAL 1/2 (of the second part) ECLAP 2012 Conference: Ingestion Process Overview

275 views

Published on

TUTORIAL ECLAP Ingestion Process Overview 1/2, professional ingestion, UGC, workflow management. Europeana integration, Europeana export. ECLAP is an aggregator for Europeana

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

TUTORIAL 1/2 (of the second part) ECLAP 2012 Conference: Ingestion Process Overview

  1. 1. ECLAP Training Meeting http://www.eclap.eu TUTORIALECLAP Ingestion Process Overview 1/2 Pierfrancesco Bellini Email: info@eclap.eu 8 May 2012, Florence, Italy 1
  2. 2. 0. Provide content ECLAP ftp site1. Provide XML metadata 4. ECLAP(your schema) Objects are 3. Publish XML produced Metadata metadata automatically Ingestion (ECLAP schema) with content from2. Create metadatamapping Service FTP site (if ECLAP present otherwise a Portal hidden empty object is created) 5. Associate content with metadata (if needed) Can be done: 1.Manually via content update 2.Automatically if content is uploaded on FTP later 2
  3. 3. 1..5 Upload content and metadata ECLAP Portal (DSI) 3
  4. 4. 8. Automatic metadata translation6. Define IPR model7. Edit/Enrich metadata ECLAP 10. Publish on europeana Portal Europeana (DSI)9. Validate whole content October 2011, Rome, Italy 4
  5. 5. 1. You upload your metadata as XML (or CSV) to the Mapping Portal2. You create the mapping of your metadata to the ECLAP Ingestion Schema3. You apply the mapping to your metadata4. You upload your content on the ECLAP ftp site5. You publish the mapped metadata to the ECLAP Portal6. DSI receive an RSS notification of the published metadata7. DSI review the XML, if we find problems (missing language, missing IPR model, …) together we can fix them and republish the metadata.8. DSI ingest some random objects on the test portal (getting content from the eclap ftp site)9. Together we can review the generated content+metadata and decide if the mapping need to be fixed or it is good to go on the main portal.10. DSI import the metadata on the main portal backoffice and start the production of the objects on the main portal11. DSI checks if there are other problems (e.g. missing files)
  6. 6.  Create an IPR model to be associated with the content (identified by the IPR model ingestion id) In the metadata mapping associate the IPR model to be used by the content However if the IPR model is not found a new one with maximum restrictions is created and it can be modified later. <month> <year>, <city>, <state> 6
  7. 7.  For all content with workflow Europeana An automatic validation rule is run after 2 month from upload to check if the minimum metadata required by europeana are present:  title or description  At least one of subject, type, coverage, spatial  language for all TEXT content  An IPR model is present In case something is missing an email is sent to the uploader In case all is OK the workflow state is set to «Under Approval» ready to be published to europeana <month> <year>, <city>, <state> 7
  8. 8. ECLAP Training Meeting http://www.eclap.eu ECLAP Ingestion Schema P. Bellini, M. Paolucci Email: info@eclap.eu October 2011, Rome, Italy 8
  9. 9.  Metadata are divided in four sections  Performing Arts  Performing arts specific elements, many are refinements of DC/DCTERMS elements  DC (Dublin Core)  The classic 15 DC elements  DCTERMS  The DCTERMS elements supported by EUROPEANA, refinements of DC elements  Technical  Elements used for the ingestion process Complete description can be found at http://bpnet.eclap.eu/drupal/?q=node/40876 (or search for “Metadata ingestion schema”) Mapping guideline  Map to more specific elements  Performing Arts, DCTERMS, DC May 2011, Athens, Greece 9
  10. 10.  FirstPerformance  Place, City, Country, Date Performance  Place, City, Country, Date PerformingArtsGroup PlotSummary Cast Professional Object (Obsolete) Genre PerformingArtType HistoricalPeriod ArtisticMovementAndActingStyle ManagementAndOrganization (Obsolete) RecordingDate PersonRecord (Obsolete, use Professional) PieceRecord ProductionRecord (Obsolete, use Professional)
  11. 11. FirstPerformance PlaceName of the theatre or venue where the performance taken place for the first time.Examples “Théâtre des Bouffes du Nord”Count 0..1Notes the first performance is the première, therefore its “place”, might not correspond with the place in which the show was recorded. For example: the opening night of “The Tragedy of Hamlet” directed by P. Brook might be held at: “Théâtre des Bouffes du Nord”, but what we are looking at on the ECLAP portal might be a video of the performance held months later - while the show was touring – at “The Globe Theatre”Refinement of DCTerms.spatial?, EDM Place
  12. 12. FirstPerformance CityName of the city where the first performance taken placeExamples “Paris”Count 0..1Notes the first performance is the première, therefore its “City”, might not correspond with the city in which the show was recorded. For example: the opening night of “The Tragedy of Hamlet” directed by P. Brook might be held in: “Paris”, but what we are looking at on the ECLAP portal might be a video of the performance held months later - while the show was touring – in “London”Refinement of DCTerms.spatial?, EDM Place
  13. 13. FirstPerformance CountryName of the country where the first performance taken placeExamples “France”Count 0..1Notes the first performance is the première, therefore its “Country”, might not correspond with the country in which the show was recorded. For example: the opening night of “The Tragedy of Hamlet” directed by P. Brook might be held in: “France”, but what we are looking at on the ECLAP portal might be a video of the performance held months later - while the show was touring – in “England”.Refinement of DCTerms.spatial
  14. 14. FirstPerformance DateDate of the first performanceExamples “2000-11-20”Count 0..1Notes the first performance is the première, therefore its “date”, might not correspond with the date in which the show was recorded. For example: the opening night of “The Tragedy of Hamlet” directed by P. Brook might be held in: “2000-11-20”, but what we are looking at on the ECLAP portal might be a video of the performance held months later, in “2001-04-05”Refinement of DCTerms.issued
  15. 15. PerformancePlacePerformanceCityPerformanceCountryPerformanceDateSimilar to the FirstPerformance… but refers to the performance represented by the digital resource (e.g. video, photo)
  16. 16. PerformingArtsGroupName of the theatre or dance company or musical group (if present)Examples “Momix”Count 0..manyRefinement of DC.creatorPlotSummarySummary of the plotExamples “Prince Hamlet mourns both his fathers death and his mother, Queen Gertrudes remarriage to Claudius. The ghost of Hamlets father appears to him and tells him that Claudius has poisoned him: Hamlet swears revenge, etc.”Count 0..manyRefinement of DC.descriptionCastName/Names of a member of the cast.Examples “Ryszard Cieślak, Rena Mirecka, Antoni Jahołkowski, Mieczysław Janowski, Maja Komorowska, Stanislaw Scierski”Count 0..manyNotes Use this element only if the Professional elements cannot be used, as the case of a cast written in a single text that cannot be easily split in all the different professional peopleRefinement of DC.contributor
  17. 17. ProfessionalA list of the people involved in the performance indicating which role each person had in the performance (eg. Actor, director, set designer etc.). It includes all the information listed in a playbill, such as the artistic cast of the show and the technicians, but also the names of the troupe which recorded the performance (eg. Cameraman, Director of Photography, etc.). Possible roles are (see next slide)Count 0..manyNotes For missing roles use Other and write the role after the name (e.g. Paolo Rossi (Writer) )Refinement of DC.contributor or DC.creator
  18. 18.  Acrobat  Musician Actor  Patron Adaptator  Performer Architect  Playwright Assistant_director  Producer Casting  Puppet_designer Choreographer  Scenographer Clown  Seamster Composer  Set_builder Costume_designer  Set_designer Critic  Singer Dancer  Sound_designer Director  Stage_manager Dramaturge  Technician Hairdresser  Theatre_manager Light_designer  Theoretician Make-up_artist  Translator Marketing_manager  Other Mask_designer Mime
  19. 19. GenreThe genre in which the work can be categorized (i.e. Ballet, Butho, Commedia dellArte, Drama, Feast Flamenco, etc)Examples “Tragedy”Count 0..manyNotes we will work on a shared vocabulary for thisRefinement of DC.subjectPerformingArtTypeType of performing art present in the content.Examples “theatre”Count 0..manyNotes identified in WP4 as cinema, dance, music, theatre, performance artRefinement of DC.type
  20. 20. HistoricalPeriodHistorical period the topic of the resource refers to.Examples “XV century”Count 0..manyRefinement of DCTerms.temporalArtisticMovementAndActingStyleArtistic movement and acting styles in which the work can be categorized (e.g. Classicism, Dada, Epic, Expressionism, etc.)Examples “Futurism”Count 0..manyNotes we will work on a shared vocabulary for thisRefinement of DC.type?
  21. 21. RecordingDateDate of creation of the digital object,Count 0..manyNotes Use this element in case what it is recorded is not a public performance (e.g. an interview) otherwise use the Performance DateRefinement of DC.datePieceRecordCredits for the text or image. The meaning of this field is a bit complex.... The text we are dealing with in this field is the script of the play. We intend this field to be filled out with the original title of the performance (eg. Medea) - which might differ from the title of the item (eg. Photo of Medea_2) - and with the name of the person who wrote the script. The records pertaining to the novel or the literary work which inspired the script should be mapped in the field "reference" instead; the field “reference” should also include the title of the novel and its author(s).Examples Title: Il Principe Costante; scenario: Jerzy Grotoski; adaptation: Julius SlowackiCount 0..manyRefinement of DCTerms.references
  22. 22.  Language  Each term can be in provided in different languages using the xml:lang attribute (use ISO 2 letters identifiers it,en,hu, …)  Language should be provided for elements that contain text that should be translated The following descriptions come from EUROPEANA ESE/EDM model description integrated with our specific considerations. EUROPEANA refers to orginal analog / born digital resource
  23. 23.  Each DC element can be repeated more times, each can have language indication  DC Contributor  DC Coverage  DC Creator  DC Date  DC Description  DC Format  DC Identifier  DC Language  DC Publisher  DC Relation  DC Rights  DC Source  DC Subject  DC Title  DC Type 23 January 2011, Florence, Italy
  24. 24. titleThe name given to the resource. Typically, a Title will be a name by which the resource is formally known. The title of the original analog or born digital object. The title should be significant and differentiate each item, so that ideally, each item should have a different title, this can be done also by adding an adjective or a number or a technical specification at the end of the title (eg. Tecniche originarie dellattore. Multicam_g.1 dx/1 or Il principe costante_sub eng)Examples “Romeo and Juliet”Count 1..manyLanguage Mandatory
  25. 25. creatorAn entity primarily responsible for making the content of the resource. Examples of a Creator include a person, an organization, or a service. Typically the name of the Creator should be used to indicate the entity. In ECLAP, the name of Partner uploading is kept automatically in a separate field. This is the name of the creator of the original analog or born digital object. This field should be used only to indicate the creator of the work of art (usually the director for a performance, the author if we are dealing with a book, the composer if we are uploading a script and so on). Often, in devised work, the creator might be the whole company or the actors might collaborate with the director. Nevertheless I guess we need to set a rule to be applied to every situation, so that I would consider actors and other artistic figures as contributors and eventually explain in the field "description" if their role as creator of the performance was capital.ExamplesCount 0..manyLanguage Optional
  26. 26. subjectThe topic of the content of the resource. Typically, a Subject will be expressed as keywords or key phrases or classification codes that describe the topic of the resource. Recommended best practice is to select a value from your own classification scheme. This is the subject of the original analog or born digital object.Count 0..manyLanguage MandatorydescriptionAn account of the content of the resource. Description may include but is not limited to: an abstract, table of contents, reference to a graphical representation of content or a free-text account of the content. A description of the original analog or born digital object.Count 0..manyLanguage Mandatory
  27. 27. publisherThe entity responsible for making the resource available. Examples of a Publisher include a person, an organization, or a service. Typically, the name of a Publisher should be used to indicate the entity. In ECLAP, the name of Partner that has provided the content is automatically tracked and stored in a different field. The name of the publisher of the original analog or born digital object. In case of a review name of the newspaper where the review was publishedExamples La RepubblicaCount 0..manyLanguage OptionalcontributorAn entity responsible for making contributions to the content of the resource. Examples of a Contributor include a person, an organization or a service. Typically, the name of a Contributor should be used to indicate the entity. In most cases, the authors of a document are listed here. The name of contributors to the original analog or born digital object. This could be a person, an organisation or a service.Count 0..manyLanguage Optional
  28. 28. dateA date associated with an event in the life cycle of the resource. Typically, Date will be associated with the creation or availability of the resource. Recommended best practice for encoding the date value is defined in a profile of ISO 8601 [Date and Time Formats, W3C Note, http://www.w3.org/TR/NOTE-datetime] and follows the YYYY-MM-DD format. If the full date is unknown, month and year (YYYY-MM) or just year (YYYY) may be used. Many other schemes are possible, but if used, they may not be easily interpreted by users or software. Use for a significant date in the life of the original analog or born digital object. Use dcterms:temporal (or dc:coverage) if the date is associated with the topic of the resource.Count 0..manyLanguage OptionaltypeThe nature or genre of the content of the resource. Type includes terms describing general categories, functions, genres, or aggregation levels for content. Recommended best practice is to select a value from a controlled vocabulary (for example, the DCMIType vocabulary http://dublincore.org/documents/dcmi-type-vocabulary/ ). To describe the physical or digital manifestation of the resource, use the FORMAT element. The type of the original analog or born digital object as recorded by the content holder, this element typically includes values such as photograph, painting, sculpture etc.Count 0..manyLanguage Mandatory
  29. 29. formatThe physical or digital manifestation of the resource. Typically, Format may include the media-type or dimensions of the resource. Examples of dimensions include size and duration. Format may be used to determine the software, hardware or other equipment needed to display or operate the resource. Recommended best practice is to select a value from a controlled vocabulary (for example, the list of Internet Media Types [http://www.iana.org/ assignments/media-types/] defining computer media formats). The unqualified element includes file format, physical medium or dimensions of the original and/or digital object. Use this element for the file format of the digital object or born digital originals. Internet Media Types [MIME] are highly recommended (http://www.iana.org/assignments/media-types/). Use of the more specific elements dcterms:extent (dimensions) and dcterms:medium (physical medium) is preferred where appropriate.Count 0..manyLanguage OptionalNotes Avoid to use it to provide the image/video format (e.g. mp4) as it may change due to transcodingidentifierAn unambiguous reference to the resource within a given context. Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system. Examples of formal identification systems include the Uniform Resource Identifier (URI) (including the Uniform Resource Locator (URL), the Digital Object Identifier (DOI) and the International Standard Book Number (ISBN). This is the identifier for the original analog or born digital object.Count 0..manyLanguage Optional
  30. 30. sourceA Reference to a resource from which the present resource is derived. The present resource may be derived from the Source resource in whole or part. Recommended best practice is to reference the resource by means of a string or number conforming to a formal identification system. In general, include in this area information about a resource that is related intellectually to the described resource but does not fit easily into a Relation element. In ECLAP, this value should be the URL or the filename of the original resource. The file uploaded and the URL provided in the upload form are tracked automatically in different fields. This element can be used for several different types of source that are related to the object (such as reference sources). The name of the content holder should no longer be recorded here as a new element.Count 0..manyLanguage OptionallanguageA language of the resource. Use ISO 639 two letter language tags (it, en, fr, de, el, …) Use this element for the language of textual objects and also where there is a language aspect to other objects e.g. sound recordings, posters, newspapers etc). If there is no language aspect to the digital object (e.g. a photograph), please ignore this element. This element is not for the language of the metadata of a resource, which may be described in xml:lang attribute. In case the digital object presents more languages, use more language elements, one for each language.Examples en, it, fr, de, el, hu, es, caCount 0..manyLanguage NoNotes Subtitles? <language>hu (subtitles)</language>relationA reference to a related resource. Recommended best practice is to reference the resource by means of a string or number conforming to a formal identification system. This is information about resources that are related to the original analog or born digital object.Count 0..manyLanguage Optional
  31. 31. coverageThe extent or scope of the content of the resource. Coverage will typically include spatial location (a place name or geographic co-ordinates), temporal period (a period label, date, or date range) or jurisdiction (such as a named administrative entity). Recommended best practice is to select a value from a controlled vocabulary (for example, the Thesaurus of Geographic Names [Getty Thesaurus of Geographic Names, http://www. getty.edu/research/tools/vocabulary/tgn/]). Where appropriate, named places or time periods should be used in preference to numeric identifiers such as sets of co-ordinates or date ranges. Coverage is the unqualified spatial or temporal coverage of the original analog or born digital object. Use of the more specific dcterms:spatial and dcterms:temporal elements is preferred where possible.Count 0..manyLanguage OptionalrightsInformation about rights held in and over the resource. Typically a Rights element will contain a rights management statement for the resource, or reference a service providing such information. Rights information often encompasses Intellectual Property Rights (IPR), Copyright, and various Property Rights. If the rights element is absent, no assumptions can be made about the status of these and other rights with respect to the resource. This is a free text element and should be used for information about intellectual property rights or access arrangements for the digital object that is additional to the controlled value provided in europeana:rights.Examples “All rights reserved”Count 0..manyLanguage Mandatory
  32. 32.  DCTERMS Alternative  DCTERMS Replaces DCTERMS Conforms To  DCTERMS Requires DCTERMS Created  DCTERMS Spatial DCTERMS Extent  DCTERMS Table of Contents DCTERMS Has Format  DCTERMS Temporal DCTERMS Has Part DCTERMS Has Version DCTERMS Is Format Of DCTERMS Is Part Of DCTERMS Is Referenced By DCTERMS Is Replaced By DCTERMS Is Required By DCTERMS Issued DCTERMS Is Version Of DCTERMS Medium DCTERMS Provenance DCTERMS References 32 January 2011, Florence, Italy
  33. 33. alternativeAn alternative name given to the resource. Typically, an Alternative title will be a name by which the resource is alternatively referred and it is different from the formal Title. Any alternative title by which the original analog or born digital object is known. This can include abbreviations or translations of the title.Count 0..manyLanguage MandatorytableOfContentsA list of subunits of the resource. A list of the units within the original analog or born digital resource object.Count 0..manyLanguage MandatorycreatedDate of creation of the resource. This is the date when the original analog or born digital object was created.Count 0..manyLanguage OptionalissuedDate of formal issuance (e.g., publication) of the resource. The date when the original analog or born digital object was issued or published.Count 0..manyLanguage Optional
  34. 34. extentThe size or duration of the resource. Refinement of format. Size or duration of the digital object and the original object may be recorded.Examples “30 pages”, “01:15:20”Count 0..manyLanguage OptionalmediumThe material or physical carrier of the resource. Refinement of dc:format.Count 0..manyLanguage OptionalspatialSpatial characteristics of the resource. Information about the spatial characteristics of the original analog or born digital object, i.e. what the resource represents or depicts in terms of space. This may be a named place, a location, a spatial coordinate or a named administrative entity.Count 0..manyLanguage OptionaltemporalTemporal characteristics of the resource. The temporal characteristics of the original analog or born digital object i.e. what the resource is about or depicts in terms of time. This may be a period, date or date range.Count 0..manyLanguage Optional
  35. 35. provenanceA statement of any changes in ownership and custody of the resource since its creation that are significant for its authenticity, integrity, and interpretation. The statement may include a description of any changes successive custodians made to the resource. This relates to the ownership and custody of the original analog or born digital object.Count 0..manyLanguage OptionalisReferencedByIs Referenced By: A related resource that references, cites, or otherwise points to the described resource.Count 0..manyLanguage OptionalreferencesA related resource that is referenced, cited, or otherwise pointed to by the described resource.Count 0..manyLanguage OptionalisPartOfIs Part Of - A related resource in which the described resource is physically or logically included. Use for the name of the collection which the digital object is part of.Count 0..manyLanguage OptionalhasPartA related resource that is included either physically or logically in the described resource. Refinement of dc:relation. See also dcterms:isPartOf.Count 0..manyLanguage Optional
  36. 36. isVersionOfA related resource of which the described resource is a version, edition, or adaptation. Changes in version imply substantive changes in content rather than differences in format. Refinement of dc:relation. See also dcterms:hasVersion.Count 0..manyLanguage OptionalhasVersionA related resource that is a version, edition, or adaptation of the described resource. Changes in version imply substantive changes in content rather than differences in format. Refinement of dc:relation. See also dcterms:isVersionOf. Use dcterms:hasFormat for differences in format.Count 0..manyLanguage OptionalisReplacedByA related resource that supplants, displaces, or supersedes the described resource.Count 0..manyLanguage OptionalreplacesA related resource that is supplanted, displaced, or superseded by the described resource.Count 0..manyLanguage Optional
  37. 37. isRequiredByA related resource that requires the described resource to support its function, delivery, or coherence.Count 0..manyLanguage OptionalrequiresA related resource that is required by the described resource to support its function, delivery, or coherence.Count 0..manyLanguage OptionalisFormatOfA related resource that is substantially the same as the described resource, but in another format. Refinement of dc:relation. See also dcterms:hasFormat.Count 0..manyLanguage OptionalhasFormatA related resource that is substantially the same as the pre-existing described resource, but in another format. Refinement of dc:relation. See also dcterms:isFormatOf. Use dcterms:hasVersion for differences in version.Count 0..manyLanguage OptionalconformsToAn established standard to which the described resource conforms. Refinement of dc:relation. The names of standards that the digital object (digitized or born digital) complies with and which are useful for the use of the object.Count 0..manyLanguage Optional
  38. 38. Information used for the ingestion process, not mapped to europeana schema. Type ProviderID ProviderName ProviderContentID ProviderContentUrl AggregationID AggregationName IPRModelID IPRContactUrl (obsolete, included in IPR Model) EuropeanaRightsUrl (obsolete, included in IPR Model)
  39. 39. Typetype of content, basic, playlist or collection BASIC_CONTENT PLAYLIST COLLECTIONExamples BASIC_CONTENTCount 1ProviderIDpartner acronym providing the data, possible values are:BEELD EN GELUID, BELLONE, CTA-UNIROMA, CTFR, ESMAE-IPP,FIFF, IKP,ITB, MUZEUM, ODIN, OSZMI, TWM, UCAM, UCLM, UG, UVA, DSI, NTUA, FRD, OTHERCount 1ProviderNameoptional name of content provider (to be used if OTHER is set as ProviderID)Count 0..1
  40. 40. ProviderContentIDidentifier of the content for the provider (used for aggregations)Examples 1267abcCount 0..1Notes Should be mandatoryProviderContentUrlurl or filename of the content, use filename if you are providing content via HardDisk or you use the ftp site (ftp.eclap.eu), in this case the file will be searched in the partner specific folder.Examples http://mysite.org/content/file.jpg, ftp://user:passw@ftp.mysite.org/content/file.pdf, file.mov, video/file.aviCount 1
  41. 41. AggregationIDidentifier that may be used to aggregate different content e.g. content related to the same performance or to the same piece.Examples Aggregation-234Count 0..1AggregationNamename of the aggregation if not provided the aggregation id is usedExamples “Shakespeare content”Count 0..1IPRModelIDfree ID identifying the IPR model to be used for the content, content with same ID will use the same IPR model. If during the ingestion an IPR model with this ID is not present a new model is automatically created with this id.Examples dsi-ipr-model-01Count 0..1Notes It should be mandatory

×