Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Assessing Compliance with the UK REF 2021 Open Access Policy

847 views

Published on

The recent increase in Open Access (OA) policies has brought forth important questions concerning the effect these policies have on the practice of publishing Open Access. In particular, is there evidence to support that mandating OA increases the proportion of OA outputs (in other words, do authors comply with relevant policies)? Furthermore, does mandating OA reduce the time from acceptance to the public availability of research outputs, and can compliance with OA mandates be effectively tracked? This work studies compliance with the UK REF 2021 Open Access policy. We use data from CrossRef and from CORE to create a dataset containing 1.6 million publications. We show that after the introduction of the UK OA policy, the proportion of OA research outputs in the UK has increased significantly, and the time lag between the acceptance of a publication and its Open Access availability has decreased, although there are significant differences in compliance between different repositories. We have developed a tool that can be used to assess publications' compliance with the policy based on a list of DOIs.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Assessing Compliance with the UK REF 2021 Open Access Policy

  1. 1. Do Authors Deposit on Time? Tracking Open Access Policy Compliance Drahomira Herrmannova Nancy Pontika Petr Knoth June 4, 2019 – JCDL 2019, Urbana-Champaign, IL Big Scientific Data and Text Analytics Group Knowledge Media Institute, The Open University
  2. 2. Introduction 1/22 https://physicstoday.scitation.org/do/10.1 063/PT.6.2.20190418a/full/ JCDL2019 Best Paper Award
  3. 3. Introduction • Why we want Open Access (OA) • Taxpayers should be able to read publicly funded research • Help researchers at poorer institutions without access to subscriptions • Institutions suffer from rising journal subscription prices • Funders introduce policies to encourage OA • Notable examples: • U.S. Public Access Plan • U.S. NIH Public Access Policy • UK REF 2021 Open Access Policy • EC H2020 Open Access Policy 2/22
  4. 4. Growing number of OA policies Source: http://roarmap.eprints.org/ Currently close to 1 thousand funder and institutional OA policies 3/22
  5. 5. OA policies • Provide criteria for making papers OA • Requirements, such as: • Where should papers be made available (publication or deposit) • When should papers be deposited • What version should be deposited (e.g. pre-print vs. post-print) • Allowed embargo periods • Etc. 4/22
  6. 6. Research questions What effect do OA policies have? 5/22
  7. 7. Research questions • Piwowar et al. (2018): At least 28% of all research papers are OA • Lariviere and Sugimoto (2018): More than two thirds of papers from selected funders (with an OA policy) were OA • Gargouri et al. (2012): OA growth often due to retroactive self- archiving (often years after publication) 6/22
  8. 8. Research questions When do author deposit? 7/22
  9. 9. Research questions When do author deposit? Do they deposit in accordance with policies? 6/22
  10. 10. Deposit time lag • What is deposit time lag? • The difference between date of publication and date of deposit in a repository expressed in days • We study deposit time lag across • Country • Time • Repository • Discipline 7/22
  11. 11. Data 8/22
  12. 12. Data 8/22
  13. 13. Data 8/22
  14. 14. Data 8/22
  15. 15. Deposit time lag calculation • Deposit time lag = deposit date – publication date • The difference was expressed in days • Positive values: article deposited after publication • Negative values: article deposited prior to publication • Best: as low value as possible 9/22
  16. 16. Dataset • 2013-2018 publications • Metadata from Crossref and CORE Publications 808,984 Repositories 728 Countries 70 Final dataset size Year of publication distribution 10/22
  17. 17. Results: Deposit time lag per country 11/22
  18. 18. Results: Deposit time lag per country/year • How has deposit time lag changed over time? • Average deposit time lag per year of publication ? 12/22
  19. 19. Results: Deposit time lag per country/year • Two options: 1. Use all data 13/22
  20. 20. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  21. 21. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2013 publications 13/22
  22. 22. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2017 publications …? 13/22
  23. 23. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  24. 24. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2013 publications 13/22
  25. 25. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2017 publications 13/22
  26. 26. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later • Underestimates deposit time lag for all, but especially for older publications 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  27. 27. Results: Deposit time lag per year/country Option 1: All data Option 2: Max deposit time lag limit (1 yr) 14/22
  28. 28. Results: Deposit time lag per subject Bars are not stacked, but overlayed 15/22
  29. 29. REF 2021 OA Policy • In 2014, the UK introduced an OA Policy for its next research assessment exercise (REF) • Requirements • Deposit final manuscript in an OA repository • Deposit on publication/acceptance or within 3 months from it • Papers published since April 2016 • Sanction • The OA requirement is linked to performance review • Did the introduction of this mandatory policy affect deposit time lag in the UK compared to other countries? 16/22
  30. 30. Single vs any repository deposit time lag 1. Single repository deposit time lag • Deposit time lag with respect to the publications’ deposit date in a given repository 2. Any repository deposit time lag • Deposit time lag with respect to the publications’ deposit date in any repository Repository 1 Repository 2 05/2017 09/2017 Single repository deposit time lag for Repository 1 = 05/2017 – publication date Any repository deposit time lag for Repository 1 = min(05/2017, 09/2017) – publication date 17/22
  31. 31. Results: UK REF compliance per year Any repository deposit time lag 18/22
  32. 32. Results: Deposit time lag per repository Full lines: Single repository deposit time lag Dashed lines: Any repository deposit time lag 19/22
  33. 33. Results: Deposit time lag per year/country Option 1: All data Option 2: Max deposit time lag limit 2014: UK introduces REF 2021 OA policy 20/22
  34. 34. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works 21/22
  35. 35. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works • Our study excludes publications that were never deposited • To quantify missing deposits we would have to correctly match all CORE publications to their Crossref metadata • Focus on deposit time lag rather than the proportion of missing deposits 21/22
  36. 36. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works • Our study excludes publications that were never deposited • To quantify missing deposits we would have to correctly match all CORE publications to their Crossref metadata • Focus on deposit time lag rather than the proportion of missing deposits • Matching between Crossref and CORE was done using metadata (titles, authors, publication years) • Strict approach, results in high accuracy (~95.27%) but lower recall 21/22
  37. 37. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset 22/22
  38. 38. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset • After introduction of the UK REF 2021 OA Policy this decrease in the UK has accelerated • As of early 2018, UK publications are deposited immediately upon publication or even slightly before 22/22
  39. 39. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset • After introduction of the UK REF 2021 OA Policy this decrease in the UK has accelerated • As of early 2018, UK publications are deposited immediately upon publication or even slightly before • Key messages: • Our observations support the argument for the inclusion of time limited deposit requirement in OA policies • Institutional practices an important role in supporting OA policy adoption 22/22
  40. 40. Thank you! Code: https://github.com/oacore/jcdl_2019 Data: https://doi.org/10.5281/zenodo.2605408

×