The document discusses a pilot study at Springer Nature to evaluate the impact of open data badges on author behavior, aimed at encouraging data sharing and increasing reader engagement. The initial findings indicate that data sharing increased significantly after the introduction of badges, although authors were often unaware of the pilot's existence and the potential effects on their behavior. Future steps include exploring automation of the badging process, expanding to more journals, and assessing different badging methodologies.

1. Do Open data badges influence
author behaviour?
A case study at Springer Nature
Rebecca Grant & Rebecca Pearce
IllustrationinspiredbytheworkofJohnMaynardKeynes

2. 1
Do Open data badges influence author behaviour?
Do Open data badges influence author behaviour?
 Context for open data badging
 The Open data badge pilot at Springer Nature
 Assessing the impact of Open data badges
 Next steps for badging

3. 2
Do Open data badges influence author behaviour?
Badging for journals: the Center for Open Science Badges to
Acknowledge Open Practices
Authors complete two disclosure items for each Open Data badge application:
 Provide the URL, doi, or other permanent path for accessing the data in a public,
open access repository
 Is there sufficient information for an independent researcher to reproduce the
reported results? If no, explain.
Peer review (certifying organisation conducts a formal review of the disclosure) can
also be undertaken

4. 3
Do Open data badges influence author behaviour?
Launched in 2014 with the journal Psychological Science
 Increase in data sharing to 23% from 3% prior to offering badges in
the first year. This increased to 39% of papers receiving badges in
the second year.
 Increase in data sharing in PS over other comparable journals
 Badges were mostly awarded on an honor system (unless
independently verified by editorial staff)
As reported in Kidwell (2016), the number of articles actually providing
available, correct, and complete was fewer than those who reported
they’d met the criteria for an Open Data Badge.
Center for Open Science Badges

5. 4
Do Open data badges influence author behaviour?
Piloting Open data badges at Springer Nature
Three key aims:
1. To assess whether Open data badges motivate authors to
share the data underpinning their published papers.
2. To assess whether reader engagement increases for
papers which are awarded a badge.
3. To establish the resources required to undertake badging
at scale.
The pilot ran from September 2018 to January 2020

6. 5
Do Open data badges influence author behaviour?
Adapting the Open data badge approach
 Self-disclosure is not required but “peer review” is used.
 Checks are made to articles’ data availability statements only.
The following criteria were used:
 A statement of data availability must be provided (data described in other parts
of the paper are not checked)
 The dataset must be deposited in a repository (closed access is acceptable, an
open licence is not required)
 A persistent identifier (e.g. DOI or accession code + repository name) must be
provided.
 The dataset is checked and must have a publicly accessible landing page, and be
the correct dataset and relevant to the research paper.

7. 6
Do Open data badges influence author behaviour?
Piloting the Open data badge

8. 7
Do Open data badges influence author behaviour?
Piloting the Open data badge
 Relatively broad-scope journal with an in-house Editorial team, which
would make coordination of the pilot more manageable.
 The journal’s data policy requires that a statement describing how data
can be accessed is included in every manuscript.
 The number of manuscripts accepted at the journal (around ten per
week) was also sufficient to allow us to gather data on the efficacy of
the pilot, but low enough to ensure that the team had sufficient time to
assess the papers, which was done manually during the pilot.

9. 8
Do Open data badges influence author behaviour?
Piloting the Open data badge
• Author is alerted to the pilot on the journal website and
manuscript submission system
• The paper, including a DAS, is published by BMC
Microbiology
• Research Data team checks the DAS for the published
paper and approves the paper for badging
• Research Data team adds the paper’s details to badgr.io;
the journal Production team add the badge to the paper

10. 9
Do Open data badges influence author behaviour?
Awarding the badges
https://doi.org/10.1186/s12866-018-1355-9

11. 10
Do Open data badges influence author behaviour?
Awarding the badges on badgr.io
https://badgr.com/public/assertions/VMxHb-
ftR_yjcgSs-n19gw

12. 11
Do Open data badges influence author behaviour?
Awarding the badges on badgr.io
https://doi.org/10.1186/s12866-018-1355-9

13. 12
Do Open data badges influence author behaviour?
12
Assessing the impact of the badges

14. 13
Do Open data badges influence author behaviour?
Refining the badge criteria: what should be badged?
• Only part of the data is shared.
• DAS refers to other parts of the paper for information (e.g. methods section).
• DAS includes a “private” repository link.
• DAS states that data in the repository will be made public when the paper is
published, but has not been made available.
• DAS including name of repository but not a PID or accession code.

15. 14
Do Open data badges influence author behaviour?
Initial pilot results (September 2018 – May 2019)
20%
31%
0%
5%
10%
15%
20%
25%
30%
35%
Pre-pilot papers Pilot papers
Papers receiving badges
n = 210n = 40

16. 15
Do Open data badges influence author behaviour?
Testing our assumptions
Were authors aware of the pilot?
Working group and Editor for journal had concerns about whether authors were
aware of the badges.
Were badges actually influencing author behavior?
Concerns were raised that journal published a lot of genetics content (a community
with mandates about data sharing), so data sharing may be due to this rather than the
badges.
Did authors know if their paper received a badge?
Authors were not being notified of badge status – were they aware of their badge?
Surveyed authors at the journal & explored other metrics related to the benefits of
badging

17. 16
Do Open data badges influence author behaviour?
In 2019 we sent a survey to every author whose paper had been assessed for an Open
data badge, with a 14% response rate.
Survey of BMC Microbiology authors
1
68
How did you learn about Badge pilot?
n=17
Learned about pilot in letters
from Editorial Manager
Website/Submission guidelines
Unaware of pilot
Only 1 author knew that
their paper had received a
badge

18. 17
Do Open data badges influence author behaviour?
Survey of BMC Microbiology authors
10
4
0
2
4
6
8
10
12
More likely No more or less likely
Would the badge make you more likely to submit to the
journal? n=14

19. 18
Do Open data badges influence author behaviour?
Other metrics: reader engagement with badged articles
53
12
0
10
20
30
40
50
60
Papers with
badges
Journal average
Average clicks to download
(published papers)
695
402
0
100
200
300
400
500
600
700
800
Papers with badges Journal average
Average unique page views

20. 19
Do Open data badges influence author behaviour?
19
Lessons Learned and Next Steps

21. 20
Do Open data badges influence author behaviour?
What we’ve learned
 There appeared to be a positive impact for papers which received an Open data
badge, in relation to access and download.
 Authors generally felt positive about badging and journals which offer badges.
 We could not be sure that authors were aware of the pilot, and that badging changed
behaviour.
 Authors are less aware of community/discipline specific data sharing mandates than
we anticipated.
 Providing a “peer reviewed” badge requires a level of data expertise and automation
would be challenging.

22. 21
Do Open data badges influence author behaviour?
Next steps
• Explore potential automation of some aspects of the badging process.
• Consider roll out to further Springer Nature journals.
• Assess the value of peer reviewed versus self-declared badging.
• Investigate pre-publication workflows and additional support for
authors wishing to comply and receive a badge.

23. 22
Do Open data badges influence author behaviour?
22
The story behind the image
John Maynard Keynes (1883–1946)
John Maynard Keynes was a British economist who
revolutionised the theory and practice of macroeconomics,
reformed economics and had a profound influence on
economic policy. This illustration represents the Keynesian
model which shows that in a monetary economy it is
possible to have periods of high unemployment unless
governments use active monetary and fiscal policy to
stimulate aggregate demand.
Thank you
Project team:
Rebecca Pearce, Journal Development Manager
rebecca.pearce@biomedcentral.com
Rebecca Grant, Research Data Manager
Rebecca.Grant@springernature.com
https://go.nature.com/ResearchDataServices
https://researchdata.springernature.com/
Image credits:
Photo by Plush Design Studio on Unsplash (slide 1)
Photo by Kimberly Farmer on Unsplash (slide 20)
Photo by Daniel von Appen on Unsplash (slide 21)
Slides are licensed as CC-BY-NC