Successfully reported this slideshow.

Citation practices and the construction of scientific fact--ECA-facts-preconference--2017-06-19

1

Share

1 of 30
1 of 30

Citation practices and the construction of scientific fact--ECA-facts-preconference--2017-06-19

1

Share

Download to read offline

Citation practices and the construction of scientific fact. Presentation at the European Conference on Argumentation preconference on status, relevance, and authority of facts.

Citation practices and the construction of scientific fact. Presentation at the European Conference on Argumentation preconference on status, relevance, and authority of facts.

More Related Content

More from jodischneider

Related Books

Free with a 14 day trial from Scribd

See all

Citation practices and the construction of scientific fact--ECA-facts-preconference--2017-06-19

  1. 1. Citation practices and the construction of scientific fact Jodi Schneider European Conference on Argumentation preconference: status, relevance, and authority of facts Fribourg, Switzerland 2017-06-19 jschneider@pobox.com http://jodischneider.com/jodi.html @jschneider
  2. 2. “[Y]ou can transform a fact into fiction or a fiction into fact just by adding or subtracting references” - Bruno Latour
  3. 3. Slide credit: Anita DeWaard: Epistemics, https://www.slideshare.net/anitawaard/epistemics/6
  4. 4. ... two miRNAs, miRNA-372 and-373, function as potential novel oncogenes in testicular germ cell tumors by inhibition of LATS2 expression, which suggests that Lats2 is an important tumor suppressor (Voorhoeve et al., 2006). Raver-Shapira et.al, JMolCell 2007 miR-372 and miR-373 target the Lats2 tumor suppressor (Voorhoeve et al., 2006) Yabuta, JBioChem 2007: As claims get cited, they become facts: To investigate the possibility that miR-372 and miR-373 suppress the expression of LATS2, we... Therefore, these results point to LATS2 as a mediator of the miR-372 and miR-373 effects on cell proliferation and tumorigenicity, Voorhoeve et al, Cell, 2006: Hypothesis Implication Cited Implication Fact Slide credit: Anita DeWaard: 'Stories that persuade with data' - talk at CENDI meeting January 9 2014 https://www.slideshare.net/anitawaard/stories-that-persuade-with-data-talk-at-cendi-meeting-january- 9-2014/6
  5. 5. Miscitation & bad assumptions “False claims regarding a causal link between game playing and obesity have propagated in the literature on exertion games.” “While the causal link between game play and obesity is not supported by evidence from health research, a version of this argument is presented many times in published exertion game literature, complete with supporting citations from public health research.” Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in exertion games literature. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 4899-4910). ACM.
  6. 6. Miscitation & bad assumptions “False claims regarding a causal link between game playing and obesity have propagated in the literature on exertion games.” “While the causal link between game play and obesity is not supported by evidence from health research, a version of this argument is presented many times in published exertion game literature, complete with supporting citations from public health research.” Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in exertion games literature. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 4899-4910). ACM.
  7. 7. How well do we cite? Haussmann, N. S., McIntyre, T., Bumby, A. J., & Loubser, M. J. (2013). Referencing practices in physical geography: how well do we cite what we write?. Progress in Physical Geography, 37(4), 543-549.
  8. 8. How might miscitation happen? • Reading errors • Hasty literature reviews • Reviewer errors • Cherry picking to justify pre-existing agenda Marshall, J., & Linehan, C. (2017). Misrepresentation of health research in exertion games literature. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 4899-4910). ACM.
  9. 9. Citing fake science harms people A paper about a clinical trial for renal disease was retracted because: “‘the trial had not been approved by the ethics committee, the involvement of a statistician could not be verified, [and] the trial was not a double-blind study, because Dr Nakao knew the treatment allocation’.” “Nevertheless, the COOPERATE study was cited by 173 review articles and 58 secondary clinical studies that enrolled a total of 35,929 patients.” “The harm done by COOPERATE is thus 4-fold: • patients were enrolled in an experimental therapy for a condition which already had an accepted therapy; • time, energy and money were wasted by patients and investigators; • false information pervaded the literature; • and combination therapy was accepted more quickly and used more widely than it might have been otherwise.” Steen, R. G. (2011). Retractions in the medical literature: how many patients are put at risk by flawed research?. Journal of Medical Ethics, 37(11), 688-692.
  10. 10. Citing fake science harms people A paper about a clinical trial for renal disease was retracted because: “‘the trial had not been approved by the ethics committee, the involvement of a statistician could not be verified, [and] the trial was not a double-blind study, because Dr Nakao knew the treatment allocation’.” “Nevertheless, the COOPERATE study was cited by 173 review articles and 58 secondary clinical studies that enrolled a total of 35,929 patients.” “The harm done by COOPERATE is thus 4-fold: • patients were enrolled in an experimental therapy for a condition which already had an accepted therapy; • time, energy and money were wasted by patients and investigators; • false information pervaded the literature; • and combination therapy was accepted more quickly and used more widely than it might have been otherwise.” Steen, R. G. (2011). Retractions in the medical literature: how many patients are put at risk by flawed research?. Journal of Medical Ethics, 37(11), 688-692.
  11. 11. “The conversion of hypothesis to fact through citation alone.” - Stephen Greenberg
  12. 12. Greenberg, Steven A. "Understanding belief using citation networks." Journal of evaluation in clinical practice 17.2 (2011): 389-393. http://dx.doi.org/ 10.1111/j.1365- 2753.2011.01646.x
  13. 13. “The conversion of hypothesis to fact through citation alone.” - Stephen Greenberg Greenberg, Steven A. "How citation distortions create unfounded authority: analysis of a citation network." BMJ 339 (2009): b2680. https://doi.org/10.1136/bmj.b2680
  14. 14. Funded grants with citation bias & citation distortion. Greenberg, Steven A. "How citation distortions create unfounded authority: analysis of a citation network." BMJ 339 (2009): b2680. https://doi.org/10.1136/bmj.b2680
  15. 15. Can we do better?
  16. 16. Boyce, R.D.: A Draft Evidence Taxonomy and Inclusion Criteria for the Drug Interaction Knowledge Base (DIKB), http://purl.net/net/drug-interaction-knowledge-base/evidence-types-and- inclusion-criteria Ask: What evidence is relevant for a given purpose?
  17. 17. Ask: What evidence is strong enough? Figure credit: SUNY Downstate Medical Center. Medical Research Library of Brooklyn. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: http://library.downstate.edu/EBM2/2100.htm
  18. 18. Ask: Can we model scientific arguments and evidence? Clark, Tim, Paolo N. Ciccarese, and Carole A. Goble. "Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications." Journal of Biomedical Semantics 5.28 (2014). http://dx.doi.org/10.1186/2041-1480-5-28
  19. 19. Jodi Schneider, Paolo Ciccarese, Tim Clark, Richard D. Boyce. “Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base.” Linked Science at ISWC 2014 http://ceur-ws.org/Vol-1282/lisc2014_submission_8.pdf
  20. 20. Modeling arguments and evidence
  21. 21. SEPIO – evidence lines Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a semantic model for the integration and analysis of scientific evidence." International Conference on Biomedical Ontology and BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf “A proposition has_evidence one or more evidence lines, which have_supporting_data one or more data items used in evaluation of the proposition’s truth.”
  22. 22. SEPIO – evidence lines example Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a semantic model for the integration and analysis of scientific evidence." International Conference on Biomedical Ontology and BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf “A simplified account of existing evidence related to this proposition is presented below, presenting summaries of five evidence lines (E1-E5) from five studies relevant to the classification of the variant for Fabry Disease: E1. Six affected individuals with the variant were found to have reduced GLA enzyme activity. E2. The variant was absent from 528 unaffected controls. E3. The variant is predicted to cause abnormal splicing that inserts additional sequence. E4. Pedigree analyses showed Fabry Disease phenotypes segregating with the variant. E5. Population databases show high frequency of individuals homozygous for the variant.”
  23. 23. SEPIO – evidence lines example Brush, Matthew, Kent Shefchek, and Melissa Haendel. "SEPIO: a semantic model for the integration and analysis of scientific evidence." International Conference on Biomedical Ontology and BioCreative. 2016. http://ceur-ws.org/Vol-1747/IT605_ICBO2016.pdf “A simplified account of existing evidence related to this proposition is presented below, presenting summaries of five evidence lines (E1-E5) from five studies relevant to the classification of the variant for Fabry Disease: E1. Six affected individuals with the variant were found to have reduced GLA enzyme activity. E2. The variant was absent from 528 unaffected controls. E3. The variant is predicted to cause abnormal splicing that inserts additional sequence. E4. Pedigree analyses showed Fabry Disease phenotypes segregating with the variant. E5. Population databases show high frequency of individuals homozygous for the variant.”
  24. 24. Modeling arguments and evidence
  25. 25. SEE Bö̈ lling, Christian, Michael Weidlich, and Hermann-Georg Holzhütter. "SEE: structured representation of scientific evidence in the biomedical domain using Semantic Web techniques." Journal of Biomedical Semantics 5.1 (2014): 1.
  26. 26. SEE Bö̈ lling, Christian, Michael Weidlich, and Hermann-Georg Holzhütter. "SEE: structured representation of scientific evidence in the biomedical domain using Semantic Web techniques." Journal of Biomedical Semantics 5.1 (2014): 1.
  27. 27. Ask: Can we structure the arguments • “An ontology is a formal, explicit specification of a shared conceptualisation.” - (Gruber, 1993) • Make clear the lines of argument • Enable formal reasoning (OWL reasoners) Gruber, Thomas R. "Toward principles for the design of ontologies used for knowledge sharing?." International journal of human-computer studies 43.5-6 (1995): 907-928. DOI:10.1006/ijhc.1995.1081
  28. 28. Ask: Should the evidence be aggregated? Figure credit: Forest plot from Underhill, Kristen, Paul Montgomery, and Don Operario. "Sexual abstinence only programmes to prevent HIV infection in high income countries: systematic review." BMJ 335.7613 (2007): 248.
  29. 29. Figure credit: Duke University Medical Center Library. Introduction to Evidence-based Practice. What is Evidence-Based Practice (EBP)? http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036021 Contextualize evidence with values and expertise

Editor's Notes

  • Greenberg, Steven A. "How citation distortions create unfounded authority: analysis of a citation network." BMJ 339 (2009): b2680. https://doi.org/10.1136/bmj.b2680
    Latour, Bruno. Science in action: How to follow scientists and engineers through society. Harvard University Press, 1987. p33
  • The COOPERATE study was a falsified clinical trial on 263 patients with non-diabetic renal disease.
  • The COOPERATE study was a falsified clinical trial on 263 patients with non-diabetic renal disease.
  • Greenberg, Steven A. "How citation distortions create unfounded authority: analysis of a citation network." BMJ 339 (2009): b2680. https://doi.org/10.1136/bmj.b2680
    Latour, Bruno. Science in action: How to follow scientists and engineers through society. Harvard University Press, 1987. p33
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “A model of the evidence for and against the assertion escitalopram does not inhibit CYP2D6. This is based on the Micropublications ontology, and reuses the ev- idence taxonomy (dikbEvidence), terms (dikb), and data from the DIKB. The Drug Ontology (DRON) and Protein Ontology (PRO) are reused in semantic qualifiers. A more detailed view of Method Me1 is shown in Figure 1. "
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • “As is commonly the case, different evidence is used by each lab - either because certain data were not accessible, or some labs judged certain data to be unreliable or irrelevant to the claim, or some labs interpreted the same data in different ways. SEPIO translates this scenario into the following narrative and set of instances to be represented in its formal modeling of the data.”
  • Gruber, Thomas R. "Toward principles for the design of ontologies used for knowledge sharing?." International journal of human-computer studies 43.5-6 (1995): 907-928.
  • Is Wikipedia a gateway to biomedical research? (Lauren Maggio)
  • ×