Monitoring a phenomenon has two remarkable effects; first it enables us to understand its properties and interact with the object or phenomenon in an informed way. The second effect (an interesting fact about social phenomenon such as publishing), is that when something is being monitored it tends to stimulate that which is being monitored. It was these facts that were the primary motivation for the Open Access Barometer – a pilot project funded by DEFF in 2013-2014. Firstly we simply didn’t know how much of the research coming out of Denmark were Open Access. Secondly we wanted to stimulate the growth of Open Access. The Danish Open Access Barometer project published a mapping of Open Access to Danish research articles and produced a prototype of a web-based Open Access barometer that through data harvest from all Danish universities could monitor the current state of Open Access (gold, green) daily and produce a number of interesting statistics including an Open Access-potential based on SHERPA/RoMEO data. In conclusion the project made a number of recommendations to monitoring Open Access and it was the hope that policy makers working with Open Access implementation would take up the idea of measuring Open Access – but we did not expect it. However, in June 2014 the Danish Minister of Higher Education and Science announced the Danish strategy for Open Access – with two remarkable goals of 80% Open Access in 2017 to publications published in 2016 and topping this by 2022 where the goal is that all (100%) publications published in 2021 should be Open Access. In order to achieve these ambitious goals a high-profile steering committee was put together. One of the key focus areas are: “The implementation of Open Access is to be monitored on an ongoing basis to ensure that all parties make a maximum effort to develop and disseminate free accessibility to Danish research findings”. To specify and ultimately measure Open Access a working group was set-up – that in its mandate was to build on the outcomes and experiences of the former DEFF project The Danish Open Access Barometer. By January 2015 this group produced a specification, price estimate and production plan for this Open Access monitor. The name was changed to Open Access indicator and will measure Open Access to Danish research from January 2016. The presenter of this contribution was project manager of the Danish Open Access Barometer and member of the Open Access indicator working group set-up by the Ministry of Higher Education and Science. Based on the Mikael K. Elbæk’s experience from this work the presentation will take you through: • Definitions – what to measures, when to measure - an imperfect compromise • Analysis and visualization – what kinds of statistics was decided to make public

1. From Open Access Barometer
to a national OA indicator
Mikael K. Elbæk
Senior Project Officer
Office for Bibliometrics and Data Management
Technical University of Denmark
@melbaek

2. Danish National Strategy on Open
Access
Announced on European Science Open
Forum (ESOF) in Copenhagen 24th of
June 2014.
By Minister for Higher Education and
Science Sofie Carsten Nielsen
Photo: NordForsk/Terje Heiestad

3. Open Access goal
2017
80%
Published in 2016
2022
100%
Published in 2021
To peer review scientific articles

4. The strategy: GREEN Open Access

5. GREEN Open Access
• No additional cost i.e. no
hybrid open access
• Negotiation with publishers
• Establishment of a national
Open Access indicator
• An OA-publishing service for
Danish Journals
Zdeněk Chalupský

6. The Open Access indicator
• Working group established by
the Ministry of Higher
Education and Science
• With the task to specify the
development of an Open
Access indicator
• In relation to the National
Danish Research Database.
• An indicator that can monitor
the implementation of the
national Open Access Strategy

7. The working group
• Mogens Sandfær, Technical University of Denmark
(chairman)
• Mikael K. Elbæk, Technical University of Denmark
• Anne Sandfær, Roskilde University
• Bertil F. Dorch, University of Southern Denmark
• Birte Christensen-Dalsgaard, The Royal Library
Representives from the ministry:
• Jonas Bak, Agency for Science and Innovation
• Hanne-Louise Kirkegaard, Agency for Science and
Innovation

8. In accordance with the mandate
• The exsisting network of CRIS/research databases
(PURE)
• The national exchange format DDF-MXD
• Harvest to the National Danish Research Database
• Include the DEFF project of renewing the National
Research Database
• Include the result of the DEFF pilot project for a
Danish Open Access Barometer
(http://www.deff.dk/aktuelt/artikel/dansk-open-
access-barometer/)

9. A manometer on a steam-engine. Manufactured by Söderströms gjuteri- och mek. verkstads A.-B. in Norrköping,
Sweden.
Photo: Zaphod Februari 6, 2005.
Danish Open Access Barometer – a pilot project

10. Vision
”To let the world know how Open Access to science
is progressing”

11. Vision
”Measuring open access will effect behaviour
towards more open access”

12. This Image was released by the United States Navy with the ID 030506-N-5862D-128
"If you can not measure it, you
can not improve it.”
Lord Kelvin

13. Strategy
• Utilize available data sources
• To visualize the (current) state of Open Access
• To demonstrate relevant and interesting data views
i.e. comparing and showing trends
• To make a user-friendly tool that can give incentives
to move Open Access forward
• Create methods and software to repeat the process
again and again.

14. Platform
• Data:
• BFI (latest dataset 2011)
• Danish National Research Database (for links to full
texts)
• SHERPA/ROMEO (for potential)
• DOAJ.org for OA-journals
• Review (to complement machine data)

15. Bibliometric Research Indicator
• Or just BFI
• B for Bibliometric
• F for Forskning = research
• I for Indicator
• Funding allocation model based on points given to
institutions based on publishing in
• A number for “expert” selected publication channels:
journals and selected publishers for books
• A common data model, all institutions have focus on
providing as correct and full data as possible, because it
is used for the allocation of funds.

17. Demarcation of data
• The data set from BFI was 38.672
• We limited to publications that has relevance to
the research funders OA-policies, i.e.:
• Peer reviewed research articles, including
• Peer reviewed artilces in conference proceedings.
• Result 16.808 records
• Peer reviewed BFI-credit giving articles alone
12.808 records

18. Two parallel tracks
Mapping of Open Access
2011
• Collecting data from
authoritative sources
• Review result, and get additions
from universities
• Analyse results
• Produce report
• Distribute for stakeholders and
decision makers
A report
Prototyping an OA
Barometer
• Use data from authoritative
sources
• Automate data collection
• To enable repetition on a frequent
basis
• Identify wanted and possible
features
• Create prototype
• Present results to stakeholders
• Document lessons learnt at use
for the next gen of the National
research database
A web site

19. The results
Mapping open access to Danish research 2011

20. Open Access baseline
• To peer review BFI-credit giving articles
11 %

21. Other types of Access

22. Delayed access

23. Hybrid Open Access

24. Some are greener than others

25. After Review
• Open Access to peer review articles
11 % 21 %

26. Open Access by type
1096, 8%
1632, 13%
10218, 79%
Golden (OA-journals)
Green (parallelpublishing)
Not-OA

27. Open Access potential
5556, 43%
7390, 57%
The total OA potential for BFI articles
No
Yes

28. Unused OA-potential
5218, 71%
2172, 29%
Total unused OA-potential for BFI articles
Yes No
Yes Yes

29. But take note! Sherpa/Romeo
data
blue, 674
gray, 983
green, 8102
white, 1621
yellow, 3993
(blank), 1435
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Total
(blank)
yellow
white
green
gray
blue

30. University overview

32. Perspectives
• Open Access metadata / vocabularies
• What kind of Open Access (other types of access)
• Identification of these OA-types
• Dates / embargoes
• Licenses
• Payments (what, when and who?)
• ORCID – to identify researchers
• FundRef and unique IDs for grants – to identify
grants and links to output

33. Publishing the data
• How open can we make the data?
• Basically we are not doing anything that a kid with
some Phyton skills could do in a day or two!
• We wish to be as OPEN as possible

34. OA Census
• Three use cases
including
• Pop in your ORCID and
get a report
• OA Hackaton 26-27
August 2013
• http://ananelson.githu
b.io/oacensus/

35. The outcome of the
National Open Access indicator working group

36. Outcome
• Overview of international experience, trends and
standards
• Analysis of the national technical and data
infrastructure including local (registration) practices
• Possible national statistics and presentation of
these
• Specification fo the technical solution, budget and
timeplan estimates

37. Main discussion points
• Definition of publication types to be measured
• Definition of what time stamp to be used
• Definition of what way Open Access can be
provided
• Definition of Open Access potential
• Definition of data views

38. Definition of publication types to be
measured
• Should conference contributions in proceedings or book
series (anthologies) be included
• Final definition:
• ”Scientific articles and conference contributions in journals
and proceedings with ISSN”.
• Close to the UK REF definition
• post-2014 REF ” The requirement applies only to journal articles
and conference proceedings with an International Standard Serial
Number” : http://www.hefce.ac.uk/pubs/year/2014/201407/

39. Definition of what time stamp to be
used
• Report (to BFI) year or Publication year or more
granular dates
• Report year was chosen after an analysis for
historical data showing that 94,9% of Report Year =
Publication year
• The benefit is that it enables the Open Access
indicator to correlated with the BFI = reuse of
deduplication service and/or analysis of BFI data.

40. Definition of what way Open Access
can be provided
• Definition in the strategy is that Open Access shall
be provided by deposit to a repository.
• First result was:
Articles deposited to local repository = Pure CRIS
Articles deposited to external subject based
repositories, on a authoritative list i.e. PubMed,
ArXiv.org etc.

41. Definition of what way Open Access
can be provided
• Questions was raised whether metadata
descriptions of publications published in OA-
journals should be included
• Result was:
An add-on to the original proposal to validate articles
published in journals was accpeted by the ministry:
Listed in DOAJ
Listed in the BFI authority lists – to ensure scientific credibility

42. Definition of Open Access potential
blue, 674
gray, 983
green, 8102
white, 1621
yellow, 3993
(blank), 1435
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Total
(blank)
yellow
white
green
gray
blue

43. Definition of Open Access potential
• In the DEFF OA-barometer pilot project the ‘green’
category alone was used as the indicator for OA-
potential
• However this will produce false-negatives
• Commissioned a survey of the actual potential
• SHERPA/RoMEO categories:
• Green = OA
• Blue = OA Potential
• Yellow ≈ likely OA Potential
• White ≠ not likely to have OA Potential
• Expected to include “Yellow” in the calculation of OA-
potential

44. Definition of data views
Three primary statistics
1. Status of national Open Access implementation
(realised OA, unused OA-potential, unclear OA-
potential)
2. A simpel status of realisation of OA-potential
3. Open Access development over time (5-years)
All presented for:
- Denmark in total
- Per university
- Per main research areas (hum, soc, sci, med)

45. Overview of the Architecture
45
☑ OAI-PMH harvest
☑ DDF-MXD (national
exchange format – with
extension)
☑ Import BFI data
☑ Import SHERPA/RoMEO
☑ Interface based on RoR
Blacklight suit
✰ DOAJ.org (not on the

46. Version 1 of the OA-indicator
46
National Styregruppe for Open Access, møde den 28. januar 2015

47. Version 1 of the OA-indicator
47
National Styregruppe for Open Access, møde den 28. januar 2015

48. Version 1 of the OA-indicator
48
National Styregruppe for Open Access, møde den 28. januar 2015

49. Version 1 of the OA-indicator
49
National Styregruppe for Open Access, møde den 28. januar 2015

50. Version 1 of the OA-indicator
50
National Styregruppe for Open Access, møde den 28. januar 2015

51. Version 1 – time plan estimates
51
National Styregruppe for Open Access, møde den 28. januar 2015
Developement
• Start in Q1-2015 in production Q1-2016
Production
• Q1-2016: First (pilot)calculation for for report year
2014
• Q1-2017: Calculation of report year 2015
• Q1-2018: Calculation of report year 2016
• Q1-2019: Calculation of report year 2017
• etc.

52. Version 2 – proposal?
52
Working group recommends :
• That a project will be initiated to analyze and make
the specification for a version 2 of the Open Access
Indicator
• To be initated after the first version has been
launched in the beginning of 2016
The working group made a rapport that stipulates some
of the initial considerations of how a version two could
look like.

53. 53

54. 54

55. 55

56. 56

57. 57

58. Lessons learned
• Policies are reflected in the definitions and workflows that
will be agreed upon = they are inclined to have
local/national quirks
• Existing infrastructures will have a huge impact on the
national solutions
• When you start to measure some people will most certainly
ask for more
• Statistics for gender/generation analysis
• Statistics for licensing negotiation
• Getting better and more data costs ressources/money!
• When politics and finance gets involved
• When monitoring Open Access becomes a political issue in
becomes a compromise / an art of the possible

59. Things we want
• Getting national licensing information into
SHERPA/RoMEO
• Discussion is started (ongoing) with SHERPA
• Open metadata from publishers
• Fuldtexts from publishers
• B2B relations between publishers and universities
that would meet marked standard i.e. financial
sector, travel sector ect.

60. The compromise
?

61. Some publishers are actually
thinking of this
http://www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf
“4th edition of the STM report on scholarly publishing”