Downloaded 306 times
Uploaded byEUDAT
Introduction to Metadata
The document provides an overview of metadata, emphasizing its importance in data discovery and understanding, as well as guidelines for creating quality metadata. It outlines types of metadata, the benefits of good practices, and the need for standardized descriptions to ensure interoperability and usability across different research communities. The document also details the EUDAT B2Find service, which facilitates metadata discovery across various data repositories.
More Related Content
Similar to Introduction to Metadata
![Polymer [ बहुलक ] Chemistry Notes PDF - Irfanullah Mehar - JJ Sir Chemistry.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/polymerchemistrynotespdf-irfanullahmehar-jjsirchemistry-260210172118-3f9b37f7-thumbnail.jpg?width=640&height=640&fit=bounds)
Introduction to Metadata
- 1. EUDAT receives fundingfrom the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065 www.eudat.eu Introduction to metadata Version 2 August 2016 This work is licensed under the Creative Commons CC-BY 4.0 licence
- 2. What is metadataand why do we need it? How to produce good quality metadata? EUDAT and metadata Overview
- 3. WHAT IS METADATA? Image CC-BY‘Metadata is a love note to the future’ by Cea+ www.flickr.com/photos/ centralasian/8071729256
- 4. Commonly defined as‘data about data’, metadata helps to make data findable and understandable Metadata can be: Descriptive: information about the content and context of the data Structural: information about the structure of the data Administrative: information about the file type, rights management and preservation processes What is metadata?
- 5. Comprehensive metadata will: Facilitatedata discovery Help users determine the applicability of the data Enable interpretation and reuse Allow any limitations to be understood Clarify ownership and restrictions on reuse Offer permanence as it transcends people and time Provide interoperability Why use metadata?
- 6. Metadata and documentation Thinkabout what will be needed in order to find, evaluate, understand, and reuse the data. Have you documented what you did and how? Did you develop code to run analyses? If so, this should be kept and shared too. Is it clear what each bit of your dataset means? Make sure the units are labelled and abbreviations explained. Record all the information needed for you and others to understand the data in the future
- 7. Information entropy The Lossof Information about Data (Metadata) Over Time, Michener et al, 1997
- 8. Create metadata atthe time of data creation Information will be forgotten and there won’t be time or effort left to capture it later. Metadata benefits from quality control at an early stage too. Time matters! Image CC-BY-SA ‘egg timer – hour glass running out’ by OpenDemocracy www.flickr.com/photos/opendemocracy/523438942
- 9. GOOD QUALITY METADATA ImageCC-BY ‘Quality’ by Elizabeth Hahn www.flickr.com/photos/128185330@N03/17517769750
- 10. Use of standards Controlledvocabularies for unambiguous keywords Simple, complete and consistent information Appropriate description Explanation of limitations to support reuse Avoid special characters e.g. !@<~ etc... Provide persistent identifiers such as DOIs What makes metadata good?
- 11. The good andthe bad Metres / seconds 2015-09-10T15:00:01+01:00 Longitudinal wind speed PDF 1.7 2008 US Population statistics Barcelona, Venezuela Furlongs and fortnight 10th Sept. 2015 15:00:01 U PDF Population statistics Barcelona More precise and standardised Ambiguous
- 12. Metadata standards Metadata standardsprovide a structured way to describe the data Information is presented in a reliable and predictable format which allows for computer interpretation Use of standards enables data interoperability
- 13. Metadata Standards Directory Catalogueinitiated by the Digital Curation Centre (DCC) now maintained as a community initiative via the Research Data Alliance www.dcc.ac.uk/resources/metadata-standards
- 14. There are anumber of factors to consider: Data type – look for standards to suit your data Community norms – what is accepted and common practice in your field? Organisational policies – is one recommended? Instruments being used – any automated metadata? What resources are available? – there are tools to create metadata in certain standards, more instructional materials and support How to choose a metadata standard?
- 15. How to writequality metadata Organise your information and reuse where possible e.g. project abstracts, lab notebooks, citations Write your metadata using a metadata tool Review for accuracy and completeness Have someone else read your record Revise based on comments from your reviewer Review once more before you publish Draft ReviewRevise Review
- 16. Tips to followwhen creating metadata Do not use jargon Define technical terms and acronyms: – CA, LA, GPS, GIS : what do these mean? Clearly state data limitations – E.g. data set omissions, completeness of data – Express considerations for appropriate re-use Use “none” or “unknown” meaningfully – None usually means that you knew about data and nothing existed (e.g., a “0” cubic feet per second discharge value) – Unknown means that you don’t know whether that data existed or not (e.g., a null value)
- 17. Dataset titles Titles arecritical in helping readers find your data – While individuals are searching for the most appropriate data sets, they are most likely going to use the title as the first criteria to determine if a dataset meets their needs. – Treat the title as the opportunity to sell your dataset. A complete title includes: What, Where, When, Who, and Scale An informative title includes: topic, timeliness of the data, specific information about place and geography
- 18. Which is thebetter title? Rivers OR Greater Yellowstone Rivers from 1:126,700 U.S. Forest Service Visitor Maps (1961-1983) Greater Yellowstone (where) Rivers (what) from 1:126,700 (scale) U.S. Forest Service (who) Visitor Maps (1961- 1983) (when)
- 19. Write for machines,not just humans Remember: a computer will read your metadata Do not use symbols that could be misinterpreted: Examples: ! @ # % { } | / < > ~ Don’t use tabs, indents, or line feeds/carriage returns When copying and pasting from other sources, use a text editor (e.g., Notepad) to eliminate hidden characters
- 20. Could someone usean automatic search to locate the data? Can others assess the usefulness of the data? Could a novice understand it? Is the metadata specific enough? Is there enough information to re-use the data? Is the information unambiguous – are all codes, abbreviations and variables explained? Remember to review your metadata!
- 21. EUDAT AND METADATA ImageCC-BY ‘University of Michigan Library Card Catalog’ by David Fulmer www.flickr.com/photos/annarbor/4350629792
- 22. B2FIND is basedon a comprehensive joint metadata catalogue of research data collections stored in EUDAT data centres and other repositories It allows researchers or data users to find relevant data, and supports communities and data providers to increase visibility of their data B2FIND provides a simple and user-friendly discovery service on metadata steadily harvested from a wide range of research communities The B2FIND service b2find.eudat.eu
- 23. The same termcan be used by different disciplines Species for chemists and zoologists Andromeda for astronomers and historians Some domain knowledge is therefore necessary The EUDAT B2FIND service needs to suit a wide range of different communities The interdisciplinary problem
- 24. Metadata is harvestedfrom different communities, usually using the OAI-PMH protocol The metadata (in a wide variety of standards) are processed to map and transform them to the B2FIND schema How the B2FIND service works INPUT Metadata in community standards e.g. DDI, Dublin Core, CMDI, ISO 19115 OUTPUT Homogenised metadata in the B2FIND schema
- 25. Metadata records inB2FIND http://b2find.eudat.eu/dataset/3a063891-6952-5bcf-a5ed-46f8a681c1c9
- 26. For more info:https://eudat.eu/services/b2find User documentation: https://www.eudat.eu/services/userdoc/b2find- integration b2find.eudat.eu
- 27. www.eudat.eu Authors Contributors This workis licensed under the Creative Commons CC-BY 4.0 licence EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065 Sarah Jones, Digital Curation Centre Shaun de Witt, STFC Sara Garavelli, Trust-IT Thank you Content has also been repurposed from the DataONE Educational modules, ‘Metadata’ and ‘How to Write Good Quality Metadata’ Retrieved from https://www.dataone.org/education-modules
Editor's Notes
- #2 This presentation will give an introduction to the concept of metadata, why it is important and how to address this in research projects.
- #3 There are three main topics that we will discuss: What is metadata and why is it important. Here we will think about the benefits that metadata can bring to you and others. Secondly, we will think about how to produce good quality metadata and offer some advice and tips To close, we’ll explain how EUDAT uses metadata in the B2FIND service
- #4 So let’s begin by thinking about what metadata is. The quote in the image here that ‘metadata is a love note to the future’ really gets at the meaning. Metadata is critical to ensure data can be found, understood and reused. If you don’t create metadata, it’s unlikely you will still understand your data in a few years time. The act of creating metadata opens up possibilities for future use.
- #5 Metadata is commonly defined as ‘data about data’. By creating metadata, you will ensure that others can find and understand your data. There are different types of metadata: Descriptive metadata includes things like the title, author, date, location, coverage and subjects. It’s all the basic information people would need to find the data and understand the main content and context. Structural metadata explains how data interrelate. For example, if a book has been digitised, you want to understand which set of images form each chapter. Administrative metadata may include information added by others e.g. preservation metadata added by a repository to note what processes have been performed on the data
- #6 There are lots of reasons to create metadata. It: Facilitates data discovery so others can find your data and your research gains more recognition and impact Good metadata helps potential users determine whether the data meet their needs, and enables them to interpret and reuse the data Metadata should outline any limitations and clarify data ownership and restrictions on reuse. This ensures others use the data appropriately Without metadata, your data will become meaningless over time as others can’t understand and reuse them. Providing associated metadata will give your data permanence and ensure they live on. By creating high quality metadata and using standards, you can also make your data interoperable
- #7 When you create metadata, it is useful to think broadly. You may come across the concept of ‘documentation’ to explain all the details you need to capture and share too. You should aim to provide all the information a third-party would need to understand and reuse your data. This may include a description of what you did, your workflows, any code created and data dictionaries or clarification of all terms and abbreviations.
- #8 This diagram from Bill Michener from DataOne in the United States shows how much information is lost over time. Metadata is a way to formalise this knowledge so your data retain meaning.
- #9 Time really matters when creating metadata – you should create metadata at the time of data creation as information is forgotten quickly. This also gives you an opportunity to do quality control early on.
- #10 We have explained why metadata is so important. Let’s now think about how to create good quality metadata.
- #11 There are lots of things you can do to improve the quality of your metadata: Primary among these is to use standards. There are lots out there so look for something relevant to your data type and discipline. Metadata standards don’t always prescribe how the information should be completed. For this you want to use controlled vocabularies or thesauri for keywords, and recognised ISO standards for common elements like languages or dates Be consistent in the information provided and ensure the description is appropriate – enough information to avoid being ambiguous but also simple and concise Any limitations with the dataset should be explained to ensure others reuse it appropriately and don’t make false assumptions You should avoid using special characters, particularly in file names or column headers in spreadsheets as some software may interpret these symbols as an operator Also provide persistent identifiers so others can reliably link to and locate your data. This helps with citation and tracking impact too.
- #12 Let’s look at some good and bad examples. You can see what we’re looking for in terms of metadata is more precise and standardised entries rather than information that could be ambiguous. Metres and seconds are universally accepted units of measurement as opposed to furlongs or fortnights For clarity, provide dates and times in the ISO standard, specifying the timezone In the third example we can see a properly described variable as opposed to an abbreviation which others may not understand When stating file formats, it is always useful to specify the version too The final two examples show the need to be specific so others can understand the coverage properly
- #13 It is highly recommended to use metadata standards. It enables interoperability and ensures the information is presented in a predictable way to allow it to be processed by computers.
- #14 There are lots of standards that can be used and you can search for them by discipline. The DCC started a catalogue of disciplinary metadata standards which is now being taken forward as an international initiative via an RDA working group
- #15 When choosing a metadata standard, you should consider: Your data type What is accepted practice for your field Whether your organisation or the tools and instruments being used suggest using one format over another (for example if one is recommended or if some metadata is created automatically in a given standard) Also think about what resources you have available. Some standards have associated tools and more comprehensive instructional materials, so they may be preferred.
- #16 When writing metadata, reuse information where possible rather than starting from scratch. For example, you may be able to use a project abstract written for your proposal, information from lab notebooks or citations for data you are reusing. Where possible, write your metadata using a tool to make the processes easier and more consistent. Think about metadata creation as an iterative process. It’s best to ask somebody else to read your record to make sure it makes sense to others and then revise and review it again before publishing.
- #17 Some general tips to follow include: Avoid using jargon Define any abbreviations, acronyms or technical terms Clearly state limitations and express considerations for appropriate reuse Use terms like ‘none’ or ‘unknown’ properly
- #18 Be comprehensive when writing your dataset title as this is how others will determine whether to look into your data further. A complete title should explain what the data relates to, a location, time period, subject and scale.
- #19 This example illustrates the importance of descriptive titles in metadata records. The second title gives enough detail for a reader to discern whether they might like more information about your data.
- #20 When you are writing your metadata, remember that it will be read by machines as well as people. You should avoid using symbols that could be misinterpreted, and tabs/indents/breaks that may be stripped out. Using a text editor for copying data will ensure hidden characters and formatting are removed.
- #21 The final point to reiterate is the need to review your metadata. It’s always useful to get a second opinion to make sure others can understand it and feel it’s clear and specific enough.
- #22 To close we want to explain how EUDAT is approaching metadata and how services like B2FIND can help you
- #23 B2FIND provides a simple and user friendly service that allows the users to discover a wide range of metadata from a variety of research communities. It is based upon a comprehensive metadata catalogue of data collections stored in the EUDAT data centres and harvested from other data repositories. The B2FIND service helps researchers to find relevant data to reuse, and helps data providers to increase the visibility of their data.
- #24 Since EUDAT is a pan-European infrastructure supporting a wide range of disciplines, we have to think about how terms are used differently by different communities. Chemical species for examples are atoms, molecules, ions etc, whereas for zoologists, ‘species’ denotes different families of animals
- #25 The B2FIND service works by harvesting metadata from different communities. This is done on a regular and incremental basis, usually using the OAI-PMH protocol. The metadata is provided in a range of community standards. It is then processed to transform it to the generic B2FIND schema to allow cross-search.
- #26 This is what a record looks like in the B2FIND catalogue. There’s a basic description, a number of keyword tags and some additional information to note the source, creator, language etc
- #27 To find out more about B2FIND or use the service, please follow the links provided.