Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20190527_Brecht Wyns & Christophe Bahim _ FAIR data maturity model


Published on

Presented by Brecht Wyns & Christophe Bahim (RDA)

during the OpenAIRE workshop "Research policy monitoring in the era of Open Science and Big Data" taking place in Ghent, Belgium on May 27th and 28th 2019

Day 1: Monitoring and Infrastructure for Open Science

Published in: Science
  • Be the first to comment

  • Be the first to like this

20190527_Brecht Wyns & Christophe Bahim _ FAIR data maturity model

  1. 1. CC BY-SA 4.0 FAIR Data Maturity Model Research policy monitoring in the era of Open Science and Big Data 27th May 2019 2019-05-27 - @resdatall 1
  2. 2. CC BY-SA 4.0 - @resdatall 2 Who knows the FAIR principles? 2019-05-27 Raise your hand
  3. 3. CC BY-SA 4.0 - @resdatall 3 Who applies the FAIR principles? 2019-05-27 Raise your hand
  4. 4. CC BY-SA 4.0 - @resdatall 4 Who measures the FAIR principles? 2019-05-27 Raise your hand
  5. 5. CC BY-SA 4.0 - @resdatall 5 The FAIR Principles 2019-05-27 FINDABLE ACCESSIBLE INTEROPERABLE REUSABLE discoverable with machine readable metadata, identifiable and locatable by means of a standard identification mechanism available and obtainable to both human and machine sufficiently described and shared with the least restrictive licences, allowing the widest reuse possible across scientific disciplines and borders, and the least cumbersome integration with other data sources both syntactically parseable and semantically understandable, allowing data exchange and reuse among scientific disciplines, researches, institutions, organisations and countries
  6. 6. CC BY-SA 4.0 FAIR data as a measure for Open Science - @resdatall 6 FAIR research data management as a way to Improve scientific research; Contribute to growth and accelerate innovation; Increase the reproducibility of research; and Better inform citizens and society about the results and value of research. Relies on a set of common principles across multiple scientific disciplines 2019-05-27 SOUNDS AMAZING! BUT…
  7. 7. CC BY-SA 4.0 FAIR data maturity model - @resdatall 72019-05-27 FAIR The principles are not strict ➔ Ambiguity ➔ Wide range of interpretations of FAIRness Different FAIR Assessment Frameworks ➔ Different metrics ➔ No comparison of results ➔ No benchmark Solution • Set of core assessment criteria for FAIRness • FAIR data maturity model & toolset • RDA recommendation • FAIR data checklist Join the RDA Working Group: RDA WG web page | GitHub
  8. 8. CC BY-SA 4.0 - @resdatall 8 Stakeholders 2019-05-27 FAIR data maturity model WG Members Chairs GO FAIR FAIR Metrics Support TARGET AUDIENCE • Researchers, data stewards, other data professionals • Data service owners, e.g. infrastructure, repositories • Organisations that manage research data • Policymakers …
  9. 9. CC BY-SA 4.0 FAIR data maturity model in the context of Open Science - @resdatall 9 1 I Solution for research policy monitoring Clear set of indicators and levels associated with them Interoperability of existing/emerging FAIR assessment frameworks Pushing data owners to the next level of FAIRness 2 I Foster innovation and societal impact Better data quality More data can be processed Clear context and provenance of data Accelerate innovation in a global digital economy Savings in money and in time 2019-05-27
  10. 10. CC BY-SA 4.0 2019-04-03 - @resdatall 10 WG methodology, timeline & scope
  11. 11. CC BY-SA 4.0 Development methodology - @resdatall 11 Bottom-up approach comprising 4 phases 1. Definition 2. Development ▪ Assessment of the four FAIR principles in four ‘strands’ ▪ Fifth ‘strand’: beyond the FAIR principles 3. Testing 4. Delivery 2019-05-27
  12. 12. CC BY-SA 4.0 Overview of the methodology - @resdatall 122019-05-27
  13. 13. CC BY-SA 4.0 Timeline - @resdatall 13 Q2Q1 Q3 Q4 Q5 Q6 M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 Today Workshop #3 [June] ▪ Presentation of results ▪ Discussion on indicators & levels Workshop #4 [September] ▪ Proposals ▪ Proposed approach towards guidelines, checklist and testing Workshop #2 [April] ▪ Approval of methodology & scope ▪ Hands-on exercise Workshop #1 [February] ▪ Introduction to the WG ▪ Existing approaches ▪ Landscaping exercise 2019-05-27 … and more to come! RDA 13th Plenary - US RDA 14th Plenary - FI
  14. 14. CC BY-SA 4.0 Scope - @resdatall 142019-05-27 Entity | Dataset and data-related aspects (e.g. algorithms, tools and workflows) Nature | Generic assessment (i.e. cross-disciplines) Time | Periodically throughout the lifecycle of the data Respondent | People with data literacy (e.g. researchers, data librarians, data stewards) Audience | Researchers, data stewards, data professionals, data service owners, organisations involved in research data and policy makers
  15. 15. CC BY-SA 4.0 2019-04-03 - @resdatall 15 In practice
  16. 16. CC BY-SA 4.0 Landscaping exercise Landscaping exercise as a starting point Analysis of existing approaches Publicly available documentation and a survey Clustering questions and options FAIR facets [e.g. F1, A2] per principle Beyond the FAIR principles [e.g. data storage] Identification of potential overlaps WG to compare questions and derive common aspects - @resdatall 162019-05-27
  17. 17. CC BY-SA 4.0 Approaches analysed So far, 11 approaches are on the radar - @resdatall 17 Approaches considered ANDS-NECTAR-RDS-FAIR data assessment tool DANS-Fairdat DANS-FAIR enough? The CSIRO 5-star Data Rating Tool FAIR Metrics questionnaire Checklist for Evaluation of Dataset Fitness for Use RDA-SHARC Evaluation FAIR evaluator Approach partially considered* Data Stewardship Wizard Approaches not considered* Big Data Readiness Support Your data: A Research Data Management Guide for Researchers *Methodologies analysed but partially/not included in the results because of questions that could not be classified 2019-05-27
  18. 18. CC BY-SA 4.0 Results of the landscaping exercise - @resdatall 18 Five slide decks classifying questions FAIR – Findable [Link] FAIR – Accessible [Link] FAIR – Interoperable [Link] FAIR – Reusable [Link] Beyond the FAIRprinciples (X) [Link] Questions, options and potential overlaps A2 metadata is accessible, even when the data are no longer available 1 Will the metadata record be available even if the data is no longer available? No Unsure Yes 2 Are the metadata accessible? F4 No Yes 5 Please provide the URL to a metadata longevity plan Overlap 7 The existence of metadata even in the absence/removal of data Example 2019-05-27
  19. 19. CC BY-SA 4.0 Towards core assessment criteria - @resdatall 19 * an indicator can be seen as a component of a Principle (e.g. F1, R1) QUESTIONS • Overlaps • Principles overused / underused • Beyond the principles OPTIONS • Binary (Y/N) • Multiple choice • Free text SCORING MECHANISM • Stars • Grade • Loading bar • None INDICATORS* • Not standardising questions • Defining indicators based on questions METRICS Definition of metrics to measure the indicators MATURITY Identification of the maturity levels Existing work Scope 2019-05-27
  20. 20. CC BY-SA 4.0 Collaborative document - @resdatall 20 TAB 1 – Introduction 2019-05-27 TAB 2 – Landscaping Exercise TAB 3 – Development TAB 4 – Outstanding issues
  21. 21. CC BY-SA 4.0 2019-04-03 - @resdatall 21 Next steps
  22. 22. CC BY-SA 4.0 Next steps Development of the core assessment criteria on Google Sheet Analysis of all the FAIR principles FAIR – Findable [Link] FAIR – Accessible [Link] FAIR – Interoperable [Link] FAIR – Reusable [Link] Online workshop #3 on indicators and maturity levels at 09:00 CEST on the 18 June 2019 at 17:00 CEST on the 18 June 2019 - @resdatall 222019-05-27
  23. 23. CC BY-SA 4.0 Resources - @resdatall 23 RDA FAIR data maturity model WG RDA FAIR data maturity model WG – Case Statement data-maturity-model-wg-case-statement RDA FAIR data maturity model WG – GitHub RDA FAIR data maturity model WG – Collaborative document teSn2cPWe_RJZG0U4Hg/edit#gid=0 RDA FAIR data maturity model WG – Mailing list 2019-05-27
  24. 24. CC BY-SA 4.0 2019-04-03 - @resdatall 24 Thank you!