This document summarizes a presentation about open research and data sharing. It discusses several drivers for data sharing, including funder requirements and cultural expectations among researchers. It also examines blockers to sharing such as concerns about data being stolen or reused without permission. The presentation argues that an overemphasis on high-impact publications and journal metrics is creating problems like hyperauthorship, reproducibility issues, and retractions. It advocates for increasing transparency through measures like preregistering trials, peer reviewing methodologies, and making data openly accessible. The goal is to overhaul how research is conducted, assessed and shared in a more open and collaborative manner.
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
So, what's it all about then? Why we share research data
1. Office of Scholarly Communication
So, what’s it all about then?
Why we share data
Jisc Research Data Network meeting
Corpus Christi College, Cambridge University
6 September 2016
Dr Danny Kingsley @dannykay68
Head of Scholarly Communication
University of Cambridge
3. Sharing data
What
Where
When
Why
How
Making data available for
other researchers
Openly accessible
repositories
As close to publication
as possible
The focus of
today’s meeting
What this talk is about –
contextualise
5. Drivers for data sharing
Image by Danny Kingsley
Funders –
return on investment
+ better quality data
6. Drivers for data sharing
Image by Danny Kingsley
Funders –
return on investment
+ better quality data
Researchers –
cultural expectations
the ‘right’ thing to do
10. ‘My data is not very interesting’
• 2005 - Professor Simon Deakin part of a team
doing research on the effects of legal reforms
to shareholder, creditor and worker rights
made their datasets available
• To date, ~50 academic papers published re-
using the datasets
• Organisations include: the International
Labour Organization,Asian Development
Bank, andTheWorld Bank
11. ‘Someone might steal/plagiarise it’
‘A second concern held by some is that a new class
of research person will emerge — people who had
nothing to do with the design and execution of the
study but use another group’s data for their own
ends, possibly stealing from the research
productivity planned by the data gatherers, or even
use the data to try to disprove what the original
investigators had posited.There is concern among
some front-line researchers that the system will be
taken over by what some researchers have
characterized as “research parasites.”’
EDITORIAL ‘Data Sharing’, Dan L. Longo, M.D., and Jeffrey M. Drazen, M.D. N Engl J
Med 2016; 374:276-277January 21, 2016 DOI: 10.1056/NEJMe1516564
12. Case study
• Cambridge is a partner institution in the Jisc
Data Asset Framework (DAF) surveys
– Contributed to survey question development
– Organised ethical clearance
– Heavily promoted the survey
– 440 responses out of the total of 1185 (37% of the
responses) came from Cambridge
13. Partners?
We broke our own rule – we did not discuss this
before-hand and assumed an arrangement that
didn’t exist. A beginners’ mistake.
We would have used a non-proprietary
repository.
14. Risk of scooping?
• And we would have waited. (To give the team
time to write a couple of papers)
https://figshare.com/articles/Data_asset_framework_DAF_survey_results_2016/3796305/2
15. The bigger problem
This is what I
am going to
discuss for the
rest of this
presentation
Researchers are
in a rat race to
stay ahead
Image by Danny Kingsley
16. Series of blogs published during July &August
https://unlockingresearch.blog.lib.cam.ac.uk/?page_id=2#OpenResearch
The Case for Open Research
17. The coin in the realm of academia
Image Flickr – Leo Reynolds
18. Journal Impact Factor
Impact Factor for 2015 is
– Number of citations in 2014 of articles published
in 2012-2013 divided by:
– Number of articles published in the journal in
2012-2013 Image by Danny Kingsley
21. San Francisco Declaration on ResearchAssessment
• Themes
– The need to eliminate the use of journal-based metrics,
such as Journal Impact Factors, in funding, appointment,
and promotion considerations;
– The need to assess research on its own merits rather than
on the basis of the journal in which the research is
published; and
– The need to capitalize on the opportunities provided by
online publication (such as relaxing unnecessary limits on
the number of words, figures, and references in articles,
and exploring new indicators of significance and impact).
• http://www.ascb.org/dora/
• >12,000 individuals & >800 organisations
22. This is one of the big problems
Image by Danny Kingsley
The insistence on the need to publish novel results in
high impact journals is creating a multitude of problems
with the scientific endeavour
28. Speaking of other ways of measuring…
http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.114.191803
This Altmetrics score of 579 is “in the top 5% of all research
outputs scored by Altmetric”
29. Blogged because of author list!
https://aps.altmetric.com/details/3997327/blogs
30. Reproducibility
Scientists are very rarely rewarded for being
right, they are rewarded for publishing in
certain journals and for getting grants.
Image by Danny Kingsley
34. Retraction
• According to RetractionWatch
http://retractionwatch.com/ there are 500-
600 retractions a year
• Only 5% of publicly available versions (non-
publisher websites) of retracted works have a
retraction statement attached
http://www.ncbi.nlm.nih.gov/pmc/articles/PM
C3411255/
35. Correlation between impact factor and retraction index.
Ferric C. Fang, and Arturo Casadevall Infect. Immun.
2011;79:3855-3859
36. Time for a change
‘Richard Smith: Another step towards the post-journal world’ BMJ blog, 12
Jul, 16
Image by Danny Kingsley
39. Peer review of methodology
http://neurochambers.blogspot.co.uk/2012/10/changing-culture-of-scientific.html
40. Increased transparency
Cell Press - redesigned methods section to help authors clearly communicate how
experiments are conducted. http://www.eurekalert.org/pub_releases/2016-08/cp-
cpt082516.php
41. Open data
• “Publicly funded research data are a public
good, produced in the public interest, which
should be made openly available with as few
restrictions as possible in a timely and
responsible manner.”
• RCUK Common Principles on Data
Policyhttp://www.rcuk.ac.uk/research/datapol
icy/
43. Putting money where their mouth is
• Wellcome Open Research
wellcomeopenresearch.org/
• Authors can “rapidly publish all outputs from
their research – everything from standard
research articles and data sets to case reports,
protocols, and null and negative results.”
44. It is all connected
• Increasing access to data is part of a much
bigger agenda to overhaul how research is
shared, assessed and ultimately practiced.
• You are part of a revolution.
Image by Danny Kingsley
In 2005 Professor Deakin and the CBR research team began creating a set of linked datasets to estimate the effects of legal reforms to shareholder, creditor and worker rights. The project has evolved, broadened and resulted in far-reaching impact thanks in large part to the willingness of Deakin and his research colleagues to make their data openly available on the web. Around 50 academic papers re-using the datasets have appeared from various scholars and organisations around the world, including high profile entities such as the International Labour Organization (ILO), Asian Development Bank, and The World Bank. As of July 2016, there are three principal datasets coding for labour laws in 117 countries between 1970 and 2013 (the CBR Labour Regulation Index), shareholder protection in 30 countries between 1990 and 2013 (the CBR Extended Shareholder Protection Index), and creditor protection in 30 countries between 1990 and 2013 (the CBR Extended Creditor Protection Index). The datasets are valued by individuals and organisations alike, as they enable researchers to track changes in labour, company and insolvency law over long periods of time for many countries. The latest version of the Law, Finance & Development project datasets can be viewed at http://dx.doi.org/10.17863/CAM.506
Deakin and the Law, Finance & Development team use an innovative research method called leximetrics, which essentially involves applying the principles of econometrics to law and while using an extensive coding methodology. In addition to making their data freely available, the team also supplies codebooks to provide explanations and legal sources for all of their coding, something no other researchers had done before for this type of analysis. The codebooks make the data transparent and the study replicable, and lend more credibility to the research results. Additionally, they are continuing to further develop the datasets on shareholder and creditor rights, so that they match the labour regulation index in terms of years and countries covered
Unsurprisingly this position raised a small storm of protest (an example is here). This was so sustained that four days later a clarification was issued, which did not include the word ‘parasites’.
The nine circles of scientific hell (with apologies to Dante and xkcd)
More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research.
The data reveal sometimes-contradictory attitudes towards reproducibility. Although 52% of those surveyed agree that there is a significant 'crisis' of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature.
Correlation between impact factor and retraction index. The 2010 journal impact factor (37) is plotted against the retraction index as a measure of the frequency of retracted articles from 2001 to 2010 (see text for details). Journals analyzed were Cell, EMBO Journal, FEMS Microbiology Letters, Infection and Immunity, Journal of Bacteriology, Journal of Biological Chemistry, Journal of Experimental Medicine, Journal of Immunology, Journal of Infectious Diseases, Journal of Virology, Lancet, Microbial Pathogenesis, Molecular Microbiology, Nature, New England Journal of Medicine, PNAS, and Science.
Registered Reports will help eliminate these bad incentives by making the results almost irrelevant in reaching editorial decisions. The philosophy of this approach is as old as the scientific method itself: If our aim is to advance knowledge then editorial decisions must be based on the rigour of the experimental design and likely replicability of the findings – andnever on how the results looked in the end.
Cortex (Neurobiological journal)