The document evaluates the openness of public knowledge bases, particularly in the context of the Nucleic Acids Research database collection, emphasizing the need for adherence to FAIR principles: Findable, Accessible, Interoperable, and Reusable. It discusses challenges in achieving true openness, such as varying data licenses and the importance of traceability and connectivity in data resources. The document proposes improvements for open science practices and highlights the role of evaluation rubrics and community standards in fostering better data sharing and usage.

1. HOW OPEN IS OPEN?
AN EVALUATION RUBRIC
FOR PUBLIC
KNOWLEDGEBASES
MELISSA HAENDEL
MARCH 28TH, 2017
@ontowonka

2. THERE ARE OVER 1500 PUBLIC
DATABASES IN NUCLEIC ACIDS
RESEARCH DATABASE COLLECTION
https://doi.org/10.1093/nar/gkw1188

3. HOW MANY OF THESE ARE TRULY OPEN?
OPENNESS IS AN NAR
REQUIREMENT, BUT …

4. WHY ARE WE STILL FAILING?

5. OPEN DATA IS FAIR DATA
http://www.nature.com/articles
/sdata201618
Findable Accessible Interoperable Reusable

6. ANATOMY OF FAIR:
FINDABLE
 persistent identifier
 rich metadata
 registered or indexed in a searchable resource
McMurry et al Identifiers for the 21st century
bit.ly/identifiers-2017

7. ANATOMY OF FAIR:
ACCESSIBLE
 (meta) data are openly retrievable by their
identifier using a standardized
communications protocol
 Metadata are accessible, even when the data
are no longer available
http://api.monarchinitiative.org/api/

8. ANATOMY OF FAIR:
INTEROPERABLE
 Use a formal, accessible, shared, and broadly
applicable language for knowledge
representation
 Define semantics of all relationships, including
cross references (hint: use the Relations
Ontology!)

9. ANATOMY OF FAIR:
INTEROPERABLE
Picking on the Personal Genome Project (thanks Sasha!)
Do you have a severe genetic disease or rare genetic trait? If so, you can
add a description for your public profile.
1. Extreme susceptibility to motion sickness. - answers pertain to this trait
2. Pyloric stenosis
3. Unusually small feet for my height

10. ANATOMY OF FAIR:
REUSABLE
 Meta(data) are described with a plurality of
accurate and relevant attributes
 Detailed provenance and use of community
standards
www.obofoundry.org
https://www.w3.org/TR/hcls-dataset/
https://peerj.com/articles/2331.pdf

11. A RUBRIC FOR EVALUATION
bit.ly/eval-rfi

12. Findable Accessible Interoperable Reusable
FAIR-TLC
Traceable Licensed Connected

13. FAIR-TLC:
TRACEABILITY
 Provenance is documented and attributed
 Contributions to the content (data, tools,
algorithms, sources, etc.) are declared
 Documentation on how to cite a record from a
source or the whole resource

14. FAIR-TLC: LICENSURE
http://peterdesmet.com/posts/analyzing-gbif-data-licenses.html
Not all data resources are free to use, derive, and
redistribute, even if they are publicly funded and
seemingly publicly available.

15. FAIR-TLC: LICENSURE
http://peterdesmet.com/posts/analyzing-gbif-data-licenses.html
Standar
d
license
171
Non-
standar
d
license
1069
No
license
10734

16. NON-STANDARD LICENSES
BURDEN SCIENCE bit.ly/reusabledata-forum

17. FAIR-TLC: CONNECTED
BECAUSE AGGREGATED != INTEGRATED

18. FAIR-TLC: CONNECTED
BECAUSE AGGREGATED != INTEGRATED
192K datasets….probably more than 38 are relevant to diabetes

19. FAIR-TLC: CONNECTED
BECAUSE AGGREGATED != INTEGRATED
Similarly, clouds do not integrate data.
http://stonebond.com/wp-content/uploads/2015/05/cloud-data-bullet-points-img.jpg

20. EVALUATING THE OPEN
SCIENCE CANDIDATES Room for
improvement
bit.ly/open-science-priz
Open imaging

21. DISCUSSION:
HOW DO WE DO BETTER?
Make the right thing the easy thing:
- Carrots:
- Tenure & promotion cycles
- Dedicated funding for increasing FAIR-
TLC
- Sticks:
- Publication requirements
- Funding requirements
- Tools:
- Tracking tools
- Documentation tools

22. ARE JOURNAL DATA SHARING
POLICIES HITTING THE MARK ?
Vasilevsky et al.
https://doi.org/10.7287/peerj.preprints.2588v1

23. TOO TINY A STICK?
Vasilevsky et al.
https://doi.org/10.7287/peerj.preprints.2588v1

24. REUSABLEDATA.ORG
Curate, evaluate, and provide guidance on
legal and effective data reuse and redistrubiton
Wanna help? Join the google group at:
Seth Carbonbit.ly/reusabledata-forum

25. THANKS TO:
JULIE MCMURRY
ANDREW SU
SETH CARBON