1. Citation practices and the
construction of scientific fact
Jodi Schneider
European Conference on Argumentation preconference: status, relevance, and
authority of facts
Fribourg, Switzerland
2017-06-19
jschneider@pobox.com
http://jodischneider.com/jodi.html
@jschneider

2. “[Y]ou can transform a fact into
fiction or a fiction into fact just by
adding or subtracting references”
- Bruno Latour

3. Slide credit: Anita DeWaard: Epistemics, https://www.slideshare.net/anitawaard/epistemics/6

4. ... two miRNAs, miRNA-372 and-373, function as potential novel oncogenes in
testicular germ cell tumors by inhibition of LATS2 expression, which suggests that
Lats2 is an important tumor suppressor (Voorhoeve et al., 2006).
Raver-Shapira et.al, JMolCell 2007
miR-372 and miR-373 target the Lats2 tumor suppressor (Voorhoeve et al., 2006)
Yabuta, JBioChem 2007:
As claims get cited, they become facts:
To investigate the possibility that miR-372 and miR-373 suppress the
expression of LATS2, we...
Therefore, these results point to LATS2 as a mediator of the miR-372 and miR-373
effects on cell proliferation and tumorigenicity,
Voorhoeve et al, Cell, 2006:
Hypothesis
Implication
Cited Implication
Fact
Slide credit: Anita DeWaard: 'Stories that persuade with data' - talk at CENDI meeting January 9 2014
https://www.slideshare.net/anitawaard/stories-that-persuade-with-data-talk-at-cendi-meeting-january-
9-2014/6

5. Miscitation & bad assumptions
“False claims regarding a causal link between
game playing and obesity have propagated in the
literature on exertion games.”
“While the causal link between game play and
obesity is not supported by evidence from health
research, a version of this argument is presented
many times in published exertion game literature,
complete with supporting citations from public
health research.”
Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in
exertion games literature. In Proceedings of the 2017 CHI Conference on
Human Factors in Computing Systems (pp. 4899-4910). ACM.

6. Miscitation & bad assumptions
“False claims regarding a causal link between
game playing and obesity have propagated in the
literature on exertion games.”
“While the causal link between game play and
obesity is not supported by evidence from health
research, a version of this argument is presented
many times in published exertion game literature,
complete with supporting citations from public
health research.”
Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in
exertion games literature. In Proceedings of the 2017 CHI Conference on
Human Factors in Computing Systems (pp. 4899-4910). ACM.

7. How well do we cite?
Haussmann, N. S., McIntyre, T., Bumby, A. J., & Loubser, M. J. (2013).
Referencing practices in physical geography: how well do we cite what we
write?. Progress in Physical Geography, 37(4), 543-549.

8. How might miscitation happen?
• Reading errors
• Hasty literature reviews
• Reviewer errors
• Cherry picking to justify pre-existing agenda
Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in
exertion games literature. In Proceedings of the 2017 CHI Conference on
Human Factors in Computing Systems (pp. 4899-4910). ACM.

9. Citing fake science harms people
A paper about a clinical trial for renal disease was retracted because:
“‘the trial had not been approved by the ethics committee, the involvement of a
statistician could not be verified, [and] the trial was not a double-blind study,
because Dr Nakao knew the treatment allocation’.”
“Nevertheless, the COOPERATE study was cited by 173 review articles and
58 secondary clinical studies that enrolled a total of 35,929 patients.”
“The harm done by COOPERATE is thus 4-fold:
• patients were enrolled in an experimental therapy for a condition which
already had an accepted therapy;
• time, energy and money were wasted by patients and investigators;
• false information pervaded the literature;
• and combination therapy was accepted more quickly and used more widely
than it might have been otherwise.”
Steen, R. G. (2011). Retractions in the medical literature: how many
patients are put at risk by flawed research?. Journal of Medical
Ethics, 37(11), 688-692.

10. Citing fake science harms people
A paper about a clinical trial for renal disease was retracted because:
“‘the trial had not been approved by the ethics committee, the involvement of a
statistician could not be verified, [and] the trial was not a double-blind study,
because Dr Nakao knew the treatment allocation’.”
“Nevertheless, the COOPERATE study was cited by 173 review articles and
58 secondary clinical studies that enrolled a total of 35,929 patients.”
“The harm done by COOPERATE is thus 4-fold:
• patients were enrolled in an experimental therapy for a condition which
already had an accepted therapy;
• time, energy and money were wasted by patients and investigators;
• false information pervaded the literature;
• and combination therapy was accepted more quickly and used more widely
than it might have been otherwise.”
Steen, R. G. (2011). Retractions in the medical literature: how many
patients are put at risk by flawed research?. Journal of Medical
Ethics, 37(11), 688-692.

11. “The conversion of hypothesis to
fact through citation alone.”
- Stephen Greenberg

12. Greenberg, Steven A.
"Understanding belief using
citation networks." Journal of
evaluation in clinical
practice 17.2 (2011): 389-393.
http://dx.doi.org/
10.1111/j.1365-
2753.2011.01646.x

13. “The conversion of hypothesis to fact
through citation alone.”
- Stephen Greenberg
Greenberg, Steven A. "How citation distortions create unfounded
authority: analysis of a citation network." BMJ 339 (2009): b2680.
https://doi.org/10.1136/bmj.b2680

14. Funded grants with citation bias &
citation distortion.
Greenberg, Steven A. "How citation distortions create unfounded
authority: analysis of a citation network." BMJ 339 (2009): b2680.
https://doi.org/10.1136/bmj.b2680

15. Can we do better?

16. Boyce, R.D.: A Draft Evidence Taxonomy and Inclusion Criteria for the
Drug Interaction Knowledge Base (DIKB),
http://purl.net/net/drug-interaction-knowledge-base/evidence-types-and-
inclusion-criteria
Ask: What evidence is relevant
for a given purpose?

17. Ask: What evidence is strong
enough?
Figure credit: SUNY Downstate Medical Center. Medical Research
Library of Brooklyn. Evidence Based Medicine Course. A Guide to
Research Methods: The Evidence Pyramid:
http://library.downstate.edu/EBM2/2100.htm

18. Ask: Can we
model scientific
arguments and
evidence?
Clark, Tim, Paolo N. Ciccarese, and Carole A. Goble.
"Micropublications: a semantic model for claims, evidence, arguments
and annotations in biomedical communications." Journal of Biomedical
Semantics 5.28 (2014). http://dx.doi.org/10.1186/2041-1480-5-28

19. Jodi Schneider, Paolo Ciccarese, Tim Clark, Richard D. Boyce. “Using the Micropublications ontology and the
Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base.” Linked
Science at ISWC 2014 http://ceur-ws.org/Vol-1282/lisc2014_submission_8.pdf

21. Modeling arguments and
evidence

22. SEPIO – evidence lines
Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a
semantic model for the integration and analysis of scientific
evidence." International Conference on Biomedical Ontology and
BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf
“A proposition has_evidence
one or more evidence lines, which have_supporting_data
one or more data items used in evaluation of the
proposition’s truth.”

23. SEPIO – evidence lines example
Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a
semantic model for the integration and analysis of scientific
evidence." International Conference on Biomedical Ontology and
BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf
“A simplified account of existing evidence related to this proposition is presented below,
presenting summaries of five evidence lines (E1-E5) from five studies relevant to the
classification of the variant for Fabry Disease:
E1. Six affected individuals with the variant were found to have reduced GLA enzyme
activity.
E2. The variant was absent from 528 unaffected controls.
E3. The variant is predicted to cause abnormal splicing that inserts additional sequence.
E4. Pedigree analyses showed Fabry Disease phenotypes segregating with the variant.
E5. Population databases show high frequency of individuals homozygous for the variant.”

24. SEPIO – evidence lines example
Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a
semantic model for the integration and analysis of scientific
evidence." International Conference on Biomedical Ontology and
BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf
“A simplified account of existing evidence related to this proposition is presented below,
presenting summaries of five evidence lines (E1-E5) from five studies relevant to the
classification of the variant for Fabry Disease:
E1. Six affected individuals with the variant were found to have reduced GLA enzyme
activity.
E2. The variant was absent from 528 unaffected controls.
E3. The variant is predicted to cause abnormal splicing that inserts additional sequence.
E4. Pedigree analyses showed Fabry Disease phenotypes segregating with the variant.
E5. Population databases show high frequency of individuals homozygous for the variant.”

25. Modeling arguments and
evidence

26. SEE
Bö̈ lling, Christian, Michael Weidlich, and Hermann-Georg Holzhütter.
"SEE: structured representation of scientific evidence in the biomedical
domain using Semantic Web techniques." Journal of Biomedical
Semantics 5.1 (2014): 1.

27. SEE
Bö̈ lling, Christian, Michael Weidlich, and Hermann-Georg Holzhütter.
"SEE: structured representation of scientific evidence in the biomedical
domain using Semantic Web techniques." Journal of Biomedical
Semantics 5.1 (2014): 1.

28. Ask: Can we structure the
arguments
• “An ontology is a formal, explicit
specification of a shared conceptualisation.”
- (Gruber, 1993)
• Make clear the lines of argument
• Enable formal reasoning (OWL reasoners)
Gruber, Thomas R. "Toward principles for the design of ontologies used
for knowledge sharing?." International journal of human-computer
studies 43.5-6 (1995): 907-928. DOI:10.1006/ijhc.1995.1081

29. Ask: Should the evidence be
aggregated?
Figure credit: Forest plot from Underhill, Kristen, Paul
Montgomery, and Don Operario. "Sexual abstinence only
programmes to prevent HIV infection in high income countries:
systematic review." BMJ 335.7613 (2007): 248.

30. Figure credit: Duke University Medical Center Library. Introduction to
Evidence-based Practice. What is Evidence-Based Practice (EBP)?
http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036021
Contextualize evidence with
values and expertise