Given at the NIH stock center directors meeting, August 8, 2016. Author: Anita Bandrowski Project: Resource Identification Initiative http://scicrunch.org/resources Topic: How is model organism data being used in literature

1. Breaking the phenotyping
barrier and the dire need
for identification
Anita Bandrowski,
SciCrunch / NIF / RRID

2. Phenotyping data comes from authors
Curators read the
articles
Align free text to
community
ontologies
Deposit aligned
data to databases
Step 1 is
identification

3. But, quality of published research is in question
Solving quality: mandates and solutions
NIH NOT-OD-16-011
Journals RRIDs
Societies FASEB Guidelines
Non-Profits TOP Guidelines
*Reproducibility of science is key*

4. What is reproducible science?
Step 1:
Materials
Step 2:
Methods

5. Step 1:
In
Practice

6. How common is this?
Papers are
currently poor at
identifying the
simplest part of
the paper, the
materials used
Vasilevsky 2013

7. But the author knows what was used!
This author got back to me within 2 hours with
the stock number of this mouse
Left open annotation with
this stock number from JAX,
but will others find this?

8. How can we get better data?
• 2009: LAMHDI meeting – project hatched
• 2011: Meeting with Society for Neuroscience, Journal of Neuroscience full editorial board presenting the
problem and results of text mining study
• 2012: Society for Neuroscience – defined the problem for Editors of top Neuroscience journals; sponsored
by INCF
• 2013: NIH Meeting - brought the editors back to define the solution; 2 day workshop sponsored by NIDA
and INCF, several IC directors in attendance
• 2013: Society for Neuroscience – mainly publishers, defined the timeline of starting the project
• 2014: Neuroscience Information Framework – built scicrunch.org/resources based on NIF technologies and
members of the OHSU team populated web pages / instructions etc.
• 2014: Project starts with Journal of Neuroscience, Neuroinformatics, F1000, Brain and Behavior and
Journal of Comparative Neurology taking a strong lead
• 2015: Paper describing how RRIDs are used by authors of the first 100 papers is co-published in 4 journals
• 2016: integration with Hypothes.is tool gives curators an easy way to verify RRIDs, sci-score gives
authors an easier way to detect what is a resource
Journals key solution
Journals will not change
Journals in aggregate
Funders role
Project Management Key
ID based tracking
Technology innovation

9. Inclusion of stock center data Repository Name Status
Ambystoma Genetic Stock Center Included
A Resource Center for Tetrahymena thermophila Included
Development of Validated Drosophila in vivo RNAi Models of Human Diseases Included
WormBase Included
JAX Included
RGD Included
MGI Included
Zebrafish International Resource Center Included
Bloomington Drosophila Stock Center at Indiana University Included
Mutant Mouse Resource and Research Center Included
MMRRC at University of California, Davis Included
ZFIN Included
Caenorhabditis Genetics Center Included
The Mouse Mutant Resource (MMR) Included
Cre Driver Strain Resources Included
Drosophila Genomics Resource Center Included
Mutant Mouse Regional Resource Center at University of North Carolina Included
Mutant Mouse Resource and Research Center at the University of Missouri Included
National Swine Resource and Research Center Included
National Xenopus Resource Center Included
Rat Resource and Research Center Included
Sperm Stem Cell Libraries for Biological Research. Included
The Special Mouse Strain Resource (SMSR) at The Jackson Laboratory Included
Xiphophorus Genetic Stock Center Included
Animal Model Resources for Cystic Fibrosis No repsonse
National Gnotobiotic Rodent Resource Center No repsonse
Adult Mesenchymal Stem Cell Resource Stocks not available
Gene Library Resource for the Sea Urchin S. purpuratus Stocks not available
Primate Embryo Gene Expression Resource Stocks not available
Research Resources for Model Amphibians Stocks not available
Viper Resource Center (VRC) at Texas A&M University-Kingsville (TAMUK) Stocks not available
WormGuides Stocks not available

10. Data in
SciCrunch is
used by
authors to
identify
organisms,
authors
publish
papers and
curators
and
scientists
then find
clear simple
citations to
resources
Copy/Paste
Publish

11. Resources are identifiable!
Bandrowski et al, 2015

12. Which repositories are being cited?
• This data set is from 980
papers, 499 organisms noted
• Curators filled in resources that
are missing RRIDs in a small
number of cases (eg.,
registered reports for cancer
reproducibility studies in eLife),
most are RRIDs asserted by
authors
• Pie chart shows total number
of organism annotations by
species
• Most citations are to
repositories that have been
included for the longest time
(eg JAX, RGD, BDSC)
• NXR – frog is a recently joined
repository, 1st paper is out
Data can be found here
https://docs.google.com/spreadsheets/d/1VOUml95YoxGQnG0hjTwOKHf6vbVb6hmNb53L2bixLGM/edit?usp=sharing
223
104
55
48
18 17
9 9 6 3 2 1 1
0
50
100
150
200
250
Fly, 55,
11%
Fish,
27, 5%
Worm,
9, 2%
Rat, 48,
10%Mouse,
359, 72%
Frog, 1,
0%
Most popular
RRID:IMSR_JAX:000664 27
RRID:RGD_70508 9
RRID:ZIRC_ZL1 7
RRID:RGD_737903 5
RRID:RGD_734476 5
RRID:IMSR_JAX:012569 5
RRID:RGD_737929 3
RRID:RGD_737891 3
RRID:IMSR_JAX:007677 4
RRID:IMSR_JAX:006410 4
RRID:IMSR_JAX:005628 4
RRID:IMSR_JAX:000671 4
RRID:ZFIN_ZDB-GENO-030619-2 3
RRID:IMSR_MMRRC:000230 3

13. Recent Journals/PublisherJuly 2016
Aug 2016 Sept/Oct
2016
11 papers

14. Why is this working?
How: Make it easy for authors to
include a unique identifier for each
research resource that is used
At publication
Instructions to authors
*Instructions to reviewers*
Direct contact from editors
We get 97% accuracy from authors
(~1000 papers verified, ~9000
RRIDs)
Bandrowski et al 2015
F1000 doi: 10.12688/f1000research.6555.2, Journal of Comparative Neurology (doi: 10.1002/cne.23913),
Brain and Behavior (doi: 10.1002/brb3.417) and NeuroInformatics (doi: 10.1007/s12021-015-9284-3).
Make compliance easy and traceable for
journals
Attention from NIH, societies
Journals listen
2014
25 journals signed on
5 executed effectively
2016
139 journals represented
10 execute effectively
*many more coming*

15. Next Steps ….we are at 1%, how to get to 100?
1.312335958
0.913705584
0.627615063
0.122019421
1.220424259
0.909090909
0
0.2
0.4
0.6
0.8
1
1.2
1.4
RRIDS AS A PERCENT OF TOTAL
PAPERS 2015/2016 (ORGANISM
SEARCH TERM)
Outreach: We need more journals / authors; we
need support from the stock centers
Peer pressure: We need to stress the new NIH
guidelines, thank you NIH!
Tools: We need to have tools that make the process
easier for journals
SciBot – finds RRIDs in papers (curation tool)
Hypothes.is – display annotations on papers – W3C
compliant annotations
Sci-Score.com – finds sentences that should have RRIDs
*estimate of the number of papers per organism in
2015/16 was based on key words in PubMed
representing the main model organism (mouse, rat etc)

17. To put the proper citation format onto your
repository!
To help increase awareness of RRIDs
(blogs, webinar, newsletters, twitter #RRID)
To add / ask for RRIDs (authors / reviewers)
To bring your journal (editors)
Comments / Complaints:
abandrowski@ucsd.edu
6 papers since late April 2016