1. Do Authors Deposit on Time?
Tracking Open Access Policy Compliance
Drahomira Herrmannova
Nancy Pontika
Petr Knoth
June 4, 2019 – JCDL 2019, Urbana-Champaign, IL
Big Scientific Data and Text Analytics Group
Knowledge Media Institute, The Open University

2. Introduction
• Why we want Open Access (OA)
• Taxpayers should be able to read publicly funded research
• Help researchers at poorer institutions without access to
subscriptions
• Institutions suffer from rising journal subscription prices
• Funders introduce policies to encourage OA
• Notable examples:
• U.S. Public Access Plan
• U.S. NIH Public Access Policy
• UK REF 2021 Open Access Policy
• EC H2020 Open Access Policy
1/22

3. Growing number of OA policies
Source: http://roarmap.eprints.org/
Currently close to 1
thousand funder and
institutional OA policies
2/22

4. OA policies
• Provide criteria for making papers OA
• Requirements, such as:
• Where should papers be made available (publication or deposit)
• When should papers be deposited
• What version should be deposited (e.g. pre-print vs. post-print)
• Allowed embargo periods
• Etc.
3/22

5. Research questions
What effect do OA policies have?
4/22

6. Research questions
• Piwowar et al. (2018): At least 28% of all research papers are OA
• Lariviere and Sugimoto (2018): More than two thirds of papers
from selected funders (with an OA policy) were OA
• Gargouri et al. (2012): OA growth often due to retroactive self-
archiving (often years after publication)
5/22

7. Research questions
When do author deposit?
6/22

8. Research questions
When do author deposit?
Do they deposit in accordance with policies?
6/22

9. Deposit time lag
• What is deposit time lag?
• The difference between date of publication and date of deposit in a
repository expressed in days
• We study deposit time lag across
• Country
• Time
• Repository
• Discipline
7/22

10. Data
8/22

11. Data
8/22

12. Data
8/22

13. Data
8/22

14. Deposit time lag calculation
• Deposit time lag = deposit
date – publication date
• The difference was expressed
in days
• Positive values: article
deposited after publication
• Negative values: article
deposited prior to
publication
• Best: as low value as possible
9/22

15. Dataset
• 2013-2018 publications
• Metadata from Crossref and CORE
Publications 808,984
Repositories 728
Countries 70
Final dataset size Year of publication distribution
10/22

16. Results: Deposit time lag per country
11/22

17. Results: Deposit time lag per country/year
• How has deposit time lag changed over time?
• Average deposit time lag per year of publication
?
12/22

18. Results: Deposit time lag per country/year
• Two options:
1. Use all data
13/22

19. Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
13/22

20. Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
Data for 2013 publications
13/22

21. Results: Deposit time lag per country/year
• Two options:
1. Use all data
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
Data for 2017 publications …?
13/22

22. Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time
lag for all, but especially for
newer publications
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
13/22

23. Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time
lag for all, but especially for
newer publications
2. Put a maximum limit on
deposit time lag for the
analysis (for comparability)
• E.g. deposit at most a year later
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
Data for 2013 publications
13/22

24. Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time
lag for all, but especially for
newer publications
2. Put a maximum limit on
deposit time lag for the
analysis (for comparability)
• E.g. deposit at most a year later
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
Data for 2017 publications
13/22

25. Results: Deposit time lag per country/year
• Two options:
1. Use all data
• Underestimates deposit time
lag for all, but especially for
newer publications
2. Put a maximum limit on
deposit time lag for the
analysis (for comparability)
• E.g. deposit at most a year later
• Underestimates deposit time
lag for all, but especially for
older publications
2011 2012 2013 2014 2015 2016 2017 2018
Yearly deposits – toy example
2013 publications 2017 publications
13/22

26. Results: Deposit time lag per year/country
Option 1: All data Option 2: Max deposit time lag limit (1 yr)
14/22

27. Results: Deposit time lag per subject
Bars are not
stacked, but
overlayed
15/22

28. REF 2021 OA Policy
• In 2014, the UK introduced an OA Policy for its next research
assessment exercise (REF)
• Requirements
• Deposit final manuscript in an OA repository
• Deposit on publication/acceptance or within 3 months from it
• Papers published since April 2016
• Sanction
• The OA requirement is linked to performance review
• Did the introduction of this mandatory policy affect deposit
time lag in the UK compared to other countries?
16/22

29. Single vs any repository deposit time lag
1. Single repository deposit time lag
• Deposit time lag with respect to the publications’ deposit date in a
given repository
2. Any repository deposit time lag
• Deposit time lag with respect to the publications’ deposit date in any
repository
Repository 1 Repository 2
05/2017 09/2017
Single repository deposit
time lag for Repository 1 =
05/2017 – publication date
Any repository deposit
time lag for Repository 1 =
min(05/2017, 09/2017) –
publication date
17/22

30. Results: UK REF compliance per year
Any repository deposit time lag
18/22

31. Results: Deposit time lag per repository
Full lines: Single repository deposit time lag
Dashed lines: Any repository deposit time lag 19/22

32. Results: Deposit time lag per year/country
Option 1: All data Option 2: Max deposit time lag limit
2014: UK introduces REF 2021 OA policy 20/22

33. Discussion
• Study assumption: if metadata deposited, then the full text is
also deposited
• Validation of full text deposits complicated due to the way the OAI-
PMH works
21/22

34. Discussion
• Study assumption: if metadata deposited, then the full text is
also deposited
• Validation of full text deposits complicated due to the way the OAI-
PMH works
• Our study excludes publications that were never deposited
• To quantify missing deposits we would have to correctly match all
CORE publications to their Crossref metadata
• Focus on deposit time lag rather than the proportion of missing
deposits
21/22

35. Discussion
• Study assumption: if metadata deposited, then the full text is
also deposited
• Validation of full text deposits complicated due to the way the OAI-
PMH works
• Our study excludes publications that were never deposited
• To quantify missing deposits we would have to correctly match all
CORE publications to their Crossref metadata
• Focus on deposit time lag rather than the proportion of missing
deposits
• Matching between Crossref and CORE was done using
metadata (titles, authors, publication years)
• Strict approach, results in high accuracy (~95.27%) but lower recall
21/22

36. Conclusions
• Time between publication and deposit has decreased
significantly in the 2013-2017 period globally
• By 472 days per country on average across all countries in our dataset
22/22

37. Conclusions
• Time between publication and deposit has decreased
significantly in the 2013-2017 period globally
• By 472 days per country on average across all countries in our dataset
• After introduction of the UK REF 2021 OA Policy this decrease in
the UK has accelerated
• As of early 2018, UK publications are deposited immediately upon
publication or even slightly before
22/22

38. Conclusions
• Time between publication and deposit has decreased
significantly in the 2013-2017 period globally
• By 472 days per country on average across all countries in our dataset
• After introduction of the UK REF 2021 OA Policy this decrease in
the UK has accelerated
• As of early 2018, UK publications are deposited immediately upon
publication or even slightly before
• Key messages:
• Our observations support the argument for the inclusion of time
limited deposit requirement in OA policies
• Institutional practices an important role in supporting OA policy
adoption
22/22

39. Thank you!
Code: https://github.com/oacore/jcdl_2019
Data: https://doi.org/10.5281/zenodo.2605408