Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Do Authors Deposit on Time?
Tracking Open Access Policy
Compliance
Drahomira Herrmannova
Nancy Pontika
Petr Knoth
June 4, ...
Introduction
1/22
https://physicstoday.scitation.org/do/10.1
063/PT.6.2.20190418a/full/
JCDL2019 Best Paper
Award
Introduction
• Why we want Open Access (OA)
• Taxpayers should be able to read publicly funded research
• Help researchers...
Growing number of OA policies
Source: http://roarmap.eprints.org/
Currently close to 1
thousand funder and
institutional O...
OA policies
• Provide criteria for making papers OA
• Requirements, such as:
• Where should papers be made available (publ...
Research questions
What effect do OA policies have?
5/22
Research questions
• Piwowar et al. (2018): At least 28% of all research papers are
OA
• Lariviere and Sugimoto (2018): Mo...
Research questions
When do author deposit?
7/22
Research questions
When do author deposit?
Do they deposit in accordance with policies?
6/22
Deposit time lag
• What is deposit time lag?
• The difference between date of publication and date of deposit in a
reposit...
Data
8/22
Data
8/22
Data
8/22
Data
8/22
Deposit time lag calculation
• Deposit time lag = deposit
date – publication date
• The difference was expressed
in days
•...
Dataset
• 2013-2018 publications
• Metadata from Crossref and CORE
Publications 808,984
Repositories 728
Countries 70
Fina...
Results: Deposit time lag per country
11/22
Results: Deposit time lag per country/year
• How has deposit time lag changed over time?
• Average deposit time lag per ye...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
13/22
Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly d...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly d...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly d...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time lag
for all, but e...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time lag
for all, but e...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time lag
for all, but e...
Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time lag
for all, but e...
Results: Deposit time lag per year/country
Option 1: All data Option 2: Max deposit time lag limit (1 yr)
14/22
Results: Deposit time lag per subject
Bars are not
stacked, but
overlayed
15/22
REF 2021 OA Policy
• In 2014, the UK introduced an OA Policy for its next research
assessment exercise (REF)
• Requirement...
Single vs any repository deposit time lag
1. Single repository deposit time lag
• Deposit time lag with respect to the pub...
Results: UK REF compliance per year
Any repository deposit time lag
18/22
Results: Deposit time lag per repository
Full lines: Single repository deposit time lag
Dashed lines: Any repository depos...
Results: Deposit time lag per year/country
Option 1: All data Option 2: Max deposit time lag limit
2014: UK introduces REF...
Discussion
• Study assumption: if metadata deposited, then the full text is also
deposited
• Validation of full text depos...
Discussion
• Study assumption: if metadata deposited, then the full text is also
deposited
• Validation of full text depos...
Discussion
• Study assumption: if metadata deposited, then the full text is also
deposited
• Validation of full text depos...
Conclusions
• Time between publication and deposit has decreased significantly in
the 2013-2017 period globally
• By 472 d...
Conclusions
• Time between publication and deposit has decreased significantly in
the 2013-2017 period globally
• By 472 d...
Conclusions
• Time between publication and deposit has decreased significantly in
the 2013-2017 period globally
• By 472 d...
Thank you!
Code: https://github.com/oacore/jcdl_2019
Data: https://doi.org/10.5281/zenodo.2605408
Upcoming SlideShare
Loading in …5
×

of

Assessing Compliance with the UK REF 2021 Open Access Policy Slide 1 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 2 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 3 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 4 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 5 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 6 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 7 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 8 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 9 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 10 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 11 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 12 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 13 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 14 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 15 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 16 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 17 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 18 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 19 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 20 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 21 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 22 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 23 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 24 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 25 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 26 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 27 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 28 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 29 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 30 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 31 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 32 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 33 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 34 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 35 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 36 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 37 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 38 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 39 Assessing Compliance with the UK REF 2021 Open Access Policy Slide 40
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

Assessing Compliance with the UK REF 2021 Open Access Policy

Download to read offline

The recent increase in Open Access (OA) policies has brought forth important questions concerning the effect these policies have on the practice of publishing Open Access. In particular, is there evidence to support that mandating OA increases the proportion of OA outputs (in other words, do authors comply with relevant policies)? Furthermore, does mandating OA reduce the time from acceptance to the public availability of research outputs, and can compliance with OA mandates be effectively tracked? This work studies compliance with the UK REF 2021 Open Access policy. We use data from CrossRef and from CORE to create a dataset containing 1.6 million publications. We show that after the introduction of the UK OA policy, the proportion of OA research outputs in the UK has increased significantly, and the time lag between the acceptance of a publication and its Open Access availability has decreased, although there are significant differences in compliance between different repositories. We have developed a tool that can be used to assess publications' compliance with the policy based on a list of DOIs.

  • Be the first to like this

Assessing Compliance with the UK REF 2021 Open Access Policy

  1. 1. Do Authors Deposit on Time? Tracking Open Access Policy Compliance Drahomira Herrmannova Nancy Pontika Petr Knoth June 4, 2019 – JCDL 2019, Urbana-Champaign, IL Big Scientific Data and Text Analytics Group Knowledge Media Institute, The Open University
  2. 2. Introduction 1/22 https://physicstoday.scitation.org/do/10.1 063/PT.6.2.20190418a/full/ JCDL2019 Best Paper Award
  3. 3. Introduction • Why we want Open Access (OA) • Taxpayers should be able to read publicly funded research • Help researchers at poorer institutions without access to subscriptions • Institutions suffer from rising journal subscription prices • Funders introduce policies to encourage OA • Notable examples: • U.S. Public Access Plan • U.S. NIH Public Access Policy • UK REF 2021 Open Access Policy • EC H2020 Open Access Policy 2/22
  4. 4. Growing number of OA policies Source: http://roarmap.eprints.org/ Currently close to 1 thousand funder and institutional OA policies 3/22
  5. 5. OA policies • Provide criteria for making papers OA • Requirements, such as: • Where should papers be made available (publication or deposit) • When should papers be deposited • What version should be deposited (e.g. pre-print vs. post-print) • Allowed embargo periods • Etc. 4/22
  6. 6. Research questions What effect do OA policies have? 5/22
  7. 7. Research questions • Piwowar et al. (2018): At least 28% of all research papers are OA • Lariviere and Sugimoto (2018): More than two thirds of papers from selected funders (with an OA policy) were OA • Gargouri et al. (2012): OA growth often due to retroactive self- archiving (often years after publication) 6/22
  8. 8. Research questions When do author deposit? 7/22
  9. 9. Research questions When do author deposit? Do they deposit in accordance with policies? 6/22
  10. 10. Deposit time lag • What is deposit time lag? • The difference between date of publication and date of deposit in a repository expressed in days • We study deposit time lag across • Country • Time • Repository • Discipline 7/22
  11. 11. Data 8/22
  12. 12. Data 8/22
  13. 13. Data 8/22
  14. 14. Data 8/22
  15. 15. Deposit time lag calculation • Deposit time lag = deposit date – publication date • The difference was expressed in days • Positive values: article deposited after publication • Negative values: article deposited prior to publication • Best: as low value as possible 9/22
  16. 16. Dataset • 2013-2018 publications • Metadata from Crossref and CORE Publications 808,984 Repositories 728 Countries 70 Final dataset size Year of publication distribution 10/22
  17. 17. Results: Deposit time lag per country 11/22
  18. 18. Results: Deposit time lag per country/year • How has deposit time lag changed over time? • Average deposit time lag per year of publication ? 12/22
  19. 19. Results: Deposit time lag per country/year • Two options: 1. Use all data 13/22
  20. 20. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  21. 21. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2013 publications 13/22
  22. 22. Results: Deposit time lag per country/year • Two options: 1. Use all data 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2017 publications …? 13/22
  23. 23. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  24. 24. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2013 publications 13/22
  25. 25. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications Data for 2017 publications 13/22
  26. 26. Results: Deposit time lag per country/year • Two options: 1. Use all data • Underestimates deposit time lag for all, but especially for newer publications 2. Put a maximum limit on deposit time lag for the analysis (for comparability) • E.g. deposit at most a year later • Underestimates deposit time lag for all, but especially for older publications 2011 2012 2013 2014 2015 2016 2017 2018 Yearly deposits – toy example 2013 publications 2017 publications 13/22
  27. 27. Results: Deposit time lag per year/country Option 1: All data Option 2: Max deposit time lag limit (1 yr) 14/22
  28. 28. Results: Deposit time lag per subject Bars are not stacked, but overlayed 15/22
  29. 29. REF 2021 OA Policy • In 2014, the UK introduced an OA Policy for its next research assessment exercise (REF) • Requirements • Deposit final manuscript in an OA repository • Deposit on publication/acceptance or within 3 months from it • Papers published since April 2016 • Sanction • The OA requirement is linked to performance review • Did the introduction of this mandatory policy affect deposit time lag in the UK compared to other countries? 16/22
  30. 30. Single vs any repository deposit time lag 1. Single repository deposit time lag • Deposit time lag with respect to the publications’ deposit date in a given repository 2. Any repository deposit time lag • Deposit time lag with respect to the publications’ deposit date in any repository Repository 1 Repository 2 05/2017 09/2017 Single repository deposit time lag for Repository 1 = 05/2017 – publication date Any repository deposit time lag for Repository 1 = min(05/2017, 09/2017) – publication date 17/22
  31. 31. Results: UK REF compliance per year Any repository deposit time lag 18/22
  32. 32. Results: Deposit time lag per repository Full lines: Single repository deposit time lag Dashed lines: Any repository deposit time lag 19/22
  33. 33. Results: Deposit time lag per year/country Option 1: All data Option 2: Max deposit time lag limit 2014: UK introduces REF 2021 OA policy 20/22
  34. 34. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works 21/22
  35. 35. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works • Our study excludes publications that were never deposited • To quantify missing deposits we would have to correctly match all CORE publications to their Crossref metadata • Focus on deposit time lag rather than the proportion of missing deposits 21/22
  36. 36. Discussion • Study assumption: if metadata deposited, then the full text is also deposited • Validation of full text deposits complicated due to the way the OAI-PMH works • Our study excludes publications that were never deposited • To quantify missing deposits we would have to correctly match all CORE publications to their Crossref metadata • Focus on deposit time lag rather than the proportion of missing deposits • Matching between Crossref and CORE was done using metadata (titles, authors, publication years) • Strict approach, results in high accuracy (~95.27%) but lower recall 21/22
  37. 37. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset 22/22
  38. 38. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset • After introduction of the UK REF 2021 OA Policy this decrease in the UK has accelerated • As of early 2018, UK publications are deposited immediately upon publication or even slightly before 22/22
  39. 39. Conclusions • Time between publication and deposit has decreased significantly in the 2013-2017 period globally • By 472 days per country on average across all countries in our dataset • After introduction of the UK REF 2021 OA Policy this decrease in the UK has accelerated • As of early 2018, UK publications are deposited immediately upon publication or even slightly before • Key messages: • Our observations support the argument for the inclusion of time limited deposit requirement in OA policies • Institutional practices an important role in supporting OA policy adoption 22/22
  40. 40. Thank you! Code: https://github.com/oacore/jcdl_2019 Data: https://doi.org/10.5281/zenodo.2605408

The recent increase in Open Access (OA) policies has brought forth important questions concerning the effect these policies have on the practice of publishing Open Access. In particular, is there evidence to support that mandating OA increases the proportion of OA outputs (in other words, do authors comply with relevant policies)? Furthermore, does mandating OA reduce the time from acceptance to the public availability of research outputs, and can compliance with OA mandates be effectively tracked? This work studies compliance with the UK REF 2021 Open Access policy. We use data from CrossRef and from CORE to create a dataset containing 1.6 million publications. We show that after the introduction of the UK OA policy, the proportion of OA research outputs in the UK has increased significantly, and the time lag between the acceptance of a publication and its Open Access availability has decreased, although there are significant differences in compliance between different repositories. We have developed a tool that can be used to assess publications' compliance with the policy based on a list of DOIs.

Views

Total views

3,191

On Slideshare

0

From embeds

0

Number of embeds

2,679

Actions

Downloads

2

Shares

0

Comments

0

Likes

0

×