SlideShare a Scribd company logo
Towards Knowledge Maintenance in Scientific
Digital Libraries with the Keystone Framework
Yuanxi Fu & Jodi Schneider
School of Information Sciences
University of Illinois at Urbana-Champaign
Presentation for JCDL 2020, Virtual, 2020-08-02
Motivation: Need for KNOWLEDGE MAINTENANCE
• When you’re not an expert, how do you judge papers?
– Recency
– Citation count
– Other heuristics
• Literature becomes obsolete but usually that’s not explicit.
• These errors can be passed on & can lead to misinterpretations.
– Most research needs information from cognate fields.
How big is retraction & citation to retraction
• Over 600,000 articles directly cite a retracted paper.
• The Retraction Watch Database lists over 19,000 retracted
publications.
• In biomedicine 94% of retracted papers have received at least
one citation, with an average citation count of 35 (Dinh, …, Schneider 2019).
Motivating Questions
• Are papers citing a retracted paper
necessarily wrong?
• Does it matter when citing authors make
use of a paper whose findings are no longer
considered valid?
• When DOES the citation matter?
• Could we selectively alert authors who cite a
retracted or abandoned paper?
Introducing the Keystone Framework
Under our framework:
1) A scientific research paper puts forward at
least one main finding, along with a
logical argument, giving reasons and
evidence to support the main finding.
2) The main finding is accepted (or not) on
the basis of the logical argument.
3) Evidence from earlier literature may be
incorporated into the argument by citing a
paper and presenting it as support, using a
citation context.
Main Finding
Arguments
support
support
Data Methods Citations …
Applying the Framework
Step 1, model the argument
Citing ArticleCited Article
Citation Context
“Many papers with
known validity problems
are highly cited [3].”
Our
paper
[3] = Bar-Ilan, J.
and Halevi, G.
2018. Temporal
characteristics
of retracted
articles
Applying the Framework:
Step 2, find the citation contexts that contribute to
the argument
Applying the Framework:
Step 3, analyze the citation context:
How many items are cited?
Citation Context
“Many papers with
known validity problems
are highly cited [3].”
Singleton
Cluster
“Digital library
applications of
argumentation theory
include argument-based
retrieval [21, 30]”
Cited Article
Citation Context
“Many papers with
known validity problems
are highly cited [3].”
[3] = Bar-Ilan, J.
and Halevi, G.
2018. Temporal
characteristics
of retracted
articles
Applying the Framework:
Step 4, analyze the cited article:
What kind of support does it give?
Main Findings Support
Applying the Keystone Framework to
Knowledge Maintenance Tasks
Case Study 1: Citing Non-reproducible Code
Assess the real impact of
citing an unreliable paper
Case Study 2: Citations Supporting One Paper’s Argument
Curate a high impact paper
(citation count > 1000) to find its
keystone citations
(de Calignon et al., 2012)
Experts & Non-experts
Step 4
Flag those articles
that are potentially
impacted.
Workflow for Assessing the Citations of an Unreliable Paper
Step 2
The domain expert
develops a list of screening
questions.
Step 3
Experts/non-
experts/text mining
tools screen target
articles using the check
list.
Step 1
The domain expert
develops a generalized
argument model.
Experts
Workflow for Targeted Curation of Important Papers
Step 1
Identify main findings of the
article from abstract,
conclusion section, and
section titles.
Step 2
Construct an argument
diagram for each main
finding.
Experts
Step 3
Align citations and
citation contexts to
components in the
argument diagrams.
Step 4
Identify and categorize
keystone citation
contexts and keystone
citations.
Results from Case Study 1
Our assessment # of papers Why? What to do?
Unaffected by the code glitch 6 Didn’t directly use the protocol.
Cited it to either support
decisions they made in their
calculations or use it as
background information.
Nothing!
Affected by the code glitch 4 Followed the protocol.
Calculations supported claims
that went into abstracts or the
conclusion section.
Authors of the 4 potentially
affected papers should
double-check their results
and either amend their
claims or document how
the claims are sustained
despite the code glitch.
Results from Case Study 2
• 51 citations in the whole paper
• 5 singleton keystone citations, including
• 3 main finding singleton keystone citations support the choice of experimental
materials
• 1 pass-through keystone citation support the choice of experimental materials
• 1 singleton main finding keystone citation support the interpretation of data
• One keystone citation cluster with 3 reference items supports the
choice of experimental method.
An ad-hoc literature review from
pass-through keystone citation 1
was used to support the choice of
PSD-95 as a synaptic marker.
Research Agenda
• Scale up: Large-scale identification of keystone citations
– Test argument-based curation approaches that have already been
automated (e.g. rhetorical-based approaches, Teufel & Kan, 2009)
– Develop text mining tools to aid manual curation and screening
• Understand citation behaviors, esp. pass-through citations
• Develop a taxonomy of validity for indicating the confidence of
a reader can have in relying on or reusing the methods and
findings of a paper
References
Bar-Ilan, J. and Halevi, G. 2018. Temporal characteristics of retracted articles. Scientometrics. 116, 3 (Jun.
2018), 1771–1783. https://doi.org/10.1007/s11192-018-2802-y
Clark, T., Ciccarese, P., & Goble, C.A. (2014). Micropublications: a semantic model for claims, evidence,
arguments and annotations in biomedical communications. Journal of Biomedical Semantics, 5, 28.
https://doi.org/10.1186/2041-1480-5-28
Dinh, L., Sarol, J., Cheng, Y., Hsiao, T., Parulian, N.N., & Schneider, J. (2019). Systematic Examination of Pre- and
Post-Retraction Citations. Proceedings of ASIST, 56(1), 390-394 https://doi.org/10.1002/pra2.35
Green, N.L. (2017). Argumentation Mining in Scientific Discourse. CMNA@ICAIL.
http://ceur-ws.org/Vol-2048/paper02.pdf
Greenberg, S.A. (2009). How citation distortions create unfounded authority: analysis of a citation network.
The BMJ, 339, b2680. https://doi.org/10.1136/bmj.b2680
Teufel, S., & Kan, M. (2009). Robust Argumentative Zoning for Sensemaking in Scholarly Documents.
NLP4DL/AT4DL, 154-170. https://doi.org/10.1007/978-3-642-23160-5_10
de Calignon, A., Polydoro, M., Suárez-Calvet, M., William, C., Adamowicz, D. H., Kopeikina, K. J., Pitstick, R.,
Sahara, N., Ashe, K. H., Carlson, G. A., Spires-Jones, T. L., & Hyman, B. T. (2012). Propagation of tau
pathology in a model of early Alzheimer's disease. Neuron, 73(4), 685–697.
https://doi.org/10.1016/j.neuron.2011.11.033
Thank You!
Appendix A
Keystone Citations Found in Case Study 2
Three Types of Keystone Citation Contexts Observed
Properties of the
keystone citation
context
Removing the citation
context would weaken the
argument supporting a
main finding
Only one
paper is
cited.
The main findings of the
cited paper(s) provide
evidence to support the
argument.
Corresponding
cited article
Singleton, main-
findings support
+ + + Main-finding
keystone citation
Cluster, main-findings
support
+ - + Main-finding
keystone citation
cluster
Singleton, pass-
through support
+ + - Pass-through
keystone citation
Three main-finding keystone
citations supported the choice of
experimental materials.
Main-finding keystone citation to
support choice of material
The fourth main-finding
keystone citation supports
the interpretation of
experimental data.
Main-finding keystone citation to
support data interpretation
An ad-hoc literature review from
pass-through keystone citation 1
was used to support the choice of
PSD-95 as a synaptic marker.
Main-finding keystone citation cluster
to support material
A main-finding keystone citation
cluster supports the choice of
experimental method.
Main-finding keystone citation cluster
to support method
Appendix B
The Keystone Framework
Concepts in the Framework
• Keystone statement: any statement whose unreliability
threatens the argument for a main finding of a paper.
• Keystone citation context: citation contexts supporting
keystone statements.
– singleton vs. cluster citation context.
• A singleton citation context cites one item, e.g. ‘[2]’
• A cluster citation context cites multiple items, e.g., ‘[2, 16]’ or ‘(DeKosky and
Scheff, 1990; Scheff and Price, 2006; Terry et al., 1991)’.
Concepts in the Framework
• Keystone statement: any statement whose unreliability
threatens the argument for a main finding of a paper.
• Keystone citation context: citation contexts supporting
keystone statements.
– Number of supporting items
• A singleton citation context cites one item, e.g. ‘[2]’
• A cluster citation context cites multiple items, e.g., ‘[2, 16]’ or
‘(DeKosky and Scheff, 1990; Scheff and Price, 2006; Terry et al., 1991)’.
Concepts in the Framework
• Keystone statement: any statement whose unreliability threatens
the argument for a main finding of a paper.
• Keystone citation context: citation contexts supporting keystone
statements.
– Strength of support
• Main-findings support, if the citation context closely relates to a main finding of the
cited item.
• Pass-through support, if support can be found within the cited item but only in an
unsupported statement or a statement referencing one or more other work(s).
• No clear support, if the citation context does not clearly relate to the cited item,
either its main findings, or other statements it makes.
Concepts in the Framework
• Keystone Statement (KS):
any statement whose
unreliability threatens the
argument for a main finding
of a paper.
Main Finding
Tau pathology results in neurodegeneration
Data
support
Method
Synaptic marker
support
Keystone Statement
XXX is a good synaptic biomarker for detecting
neurodegeneration
Concepts in the Framework
• Keystone Citation Context
(KCC): citation contexts
that supports keystone
statements.
Keystone Statement
XXX is a good synaptic biomarker for detecting
neurodegeneration
Keystone Citation Context
support
Keystone Citation Contexts Continued
How many items are cited?
Singleton
Cluster
Whether the cited item’s
main findings support the
citation context?
Main-finding
support
Pass-through
support
No
support
Distinction between KS and KCC
• KSs are summation of KCCs. Domain
experts can distill the same KS from
several KCCs found in different papers.
• One study found that several citation
contexts can be distilled to a statement
about β amyloid accumulation, but the
cited papers mentioned the statement
only as “hypothesis” or not at all.
Greenberg, 2009
Statement: The accumulation of β amyloid occurs early and
precedes other abnormalities.
Citing Article Cited Article
The appearance of Aβ-
positive, noncongophilic
deposits precedes
vacuolization in IBM
muscle fibers.8 (as fact)
Some muscle fibers had
Aβ-positive
accumulations,… Those
muscle fibers,…, may
represent early changes of
IBM. (as hypothesis)
support
Appendix C
Argument-based Curation
(Teufel & Kan, 2011)
Rhetoric-based Approaches
• Extract information based on
rhetorical feature
• Limited need for domain
knowledge
• Provide a coarser argumentative
structure
• Well-automated
Argument-scheme based Approaches
Premises:
• A group of individuals G have atypical
phenotype P
• All of the individuals in G have atypical
genotype M
• Another group of individuals (controls) do not
have P
• None of controls have M
Conclusion: M may be the cause of P (in G)
(Green, 2017)
• Require deep domain analysis
• Show the logic of how a discipline
justifies its findings
• Identify potential weakness through
critical questions
• Currently depend on manual curation,
but potentially automatable through
text mining
Provenance-based Method
(Clark et al., 2014)
• Model scientists’ work process
• Most suitable for modeling
empirical research articles
• Require manual curation

More Related Content

What's hot

Data reuse and scholarly reward: understanding practice and building infrastr...
Data reuse and scholarly reward: understanding practice and building infrastr...Data reuse and scholarly reward: understanding practice and building infrastr...
Data reuse and scholarly reward: understanding practice and building infrastr...
Todd Vision
 
Answering More Questions with Provenance and Query Patterns
Answering More Questions with Provenance and Query PatternsAnswering More Questions with Provenance and Query Patterns
Answering More Questions with Provenance and Query Patterns
Bertram Ludäscher
 
An introduction to Statistical Analysis Plans
An introduction to Statistical Analysis PlansAn introduction to Statistical Analysis Plans
An introduction to Statistical Analysis Plans
University of Liverpool Library
 
Anonymisation 101
Anonymisation 101Anonymisation 101
Research data and scholarly publications: going from casual acquaintances to ...
Research data and scholarly publications: going from casual acquaintances to ...Research data and scholarly publications: going from casual acquaintances to ...
Research data and scholarly publications: going from casual acquaintances to ...
Todd Vision
 
The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]
Impactstory Team
 
Leveraging publication metadata to help overcome the data ingest bottleneck
Leveraging publication metadata to help overcome the data ingest bottleneck Leveraging publication metadata to help overcome the data ingest bottleneck
Leveraging publication metadata to help overcome the data ingest bottleneck
Todd Vision
 
Research Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific MisconductResearch Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific Misconduct
Nader Ale Ebrahim
 
Fact Checking from Text
Fact Checking from TextFact Checking from Text
Fact Checking from Text
Subhabrata Mukherjee
 
Digital Scholar Webinar: Open reproducible research
Digital Scholar Webinar: Open reproducible researchDigital Scholar Webinar: Open reproducible research
Digital Scholar Webinar: Open reproducible research
SC CTSI at USC and CHLA
 
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
jodischneider
 
The Dryad Digital Repository: Published evolutionary data as part of the gre...
The Dryad Digital Repository: Published evolutionary data as part of the gre...The Dryad Digital Repository: Published evolutionary data as part of the gre...
The Dryad Digital Repository: Published evolutionary data as part of the gre...
Todd Vision
 
Collaborative Research: Scopus & RefWorks
Collaborative Research: Scopus & RefWorksCollaborative Research: Scopus & RefWorks
Collaborative Research: Scopus & RefWorks
Robin Featherstone
 
Improving the quality of my journal and increase its impact factor
Improving the quality of my journal and increase its impact factorImproving the quality of my journal and increase its impact factor
Improving the quality of my journal and increase its impact factor
Portal de Periódicos UFSC
 
Developing a Replicable Methodology for Automated Identification of Emerging ...
Developing a Replicable Methodology for Automated Identification of Emerging ...Developing a Replicable Methodology for Automated Identification of Emerging ...
Developing a Replicable Methodology for Automated Identification of Emerging ...
University of Michigan Taubman Health Sciences Library
 
Digital Scholar Webinar: Transparent, Open, and Reproducible Research
Digital Scholar Webinar: Transparent, Open, and Reproducible ResearchDigital Scholar Webinar: Transparent, Open, and Reproducible Research
Digital Scholar Webinar: Transparent, Open, and Reproducible Research
SC CTSI at USC and CHLA
 
Sense About Science: Peer Review Survey
Sense About Science: Peer Review SurveySense About Science: Peer Review Survey
Sense About Science: Peer Review Survey
SenseAboutScience
 
Research Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific MisconductResearch Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific Misconduct
Nader Ale Ebrahim
 

What's hot (18)

Data reuse and scholarly reward: understanding practice and building infrastr...
Data reuse and scholarly reward: understanding practice and building infrastr...Data reuse and scholarly reward: understanding practice and building infrastr...
Data reuse and scholarly reward: understanding practice and building infrastr...
 
Answering More Questions with Provenance and Query Patterns
Answering More Questions with Provenance and Query PatternsAnswering More Questions with Provenance and Query Patterns
Answering More Questions with Provenance and Query Patterns
 
An introduction to Statistical Analysis Plans
An introduction to Statistical Analysis PlansAn introduction to Statistical Analysis Plans
An introduction to Statistical Analysis Plans
 
Anonymisation 101
Anonymisation 101Anonymisation 101
Anonymisation 101
 
Research data and scholarly publications: going from casual acquaintances to ...
Research data and scholarly publications: going from casual acquaintances to ...Research data and scholarly publications: going from casual acquaintances to ...
Research data and scholarly publications: going from casual acquaintances to ...
 
The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]The Right Metrics for Generation Open [Open Access Week 2014]
The Right Metrics for Generation Open [Open Access Week 2014]
 
Leveraging publication metadata to help overcome the data ingest bottleneck
Leveraging publication metadata to help overcome the data ingest bottleneck Leveraging publication metadata to help overcome the data ingest bottleneck
Leveraging publication metadata to help overcome the data ingest bottleneck
 
Research Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific MisconductResearch Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific Misconduct
 
Fact Checking from Text
Fact Checking from TextFact Checking from Text
Fact Checking from Text
 
Digital Scholar Webinar: Open reproducible research
Digital Scholar Webinar: Open reproducible researchDigital Scholar Webinar: Open reproducible research
Digital Scholar Webinar: Open reproducible research
 
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...
 
The Dryad Digital Repository: Published evolutionary data as part of the gre...
The Dryad Digital Repository: Published evolutionary data as part of the gre...The Dryad Digital Repository: Published evolutionary data as part of the gre...
The Dryad Digital Repository: Published evolutionary data as part of the gre...
 
Collaborative Research: Scopus & RefWorks
Collaborative Research: Scopus & RefWorksCollaborative Research: Scopus & RefWorks
Collaborative Research: Scopus & RefWorks
 
Improving the quality of my journal and increase its impact factor
Improving the quality of my journal and increase its impact factorImproving the quality of my journal and increase its impact factor
Improving the quality of my journal and increase its impact factor
 
Developing a Replicable Methodology for Automated Identification of Emerging ...
Developing a Replicable Methodology for Automated Identification of Emerging ...Developing a Replicable Methodology for Automated Identification of Emerging ...
Developing a Replicable Methodology for Automated Identification of Emerging ...
 
Digital Scholar Webinar: Transparent, Open, and Reproducible Research
Digital Scholar Webinar: Transparent, Open, and Reproducible ResearchDigital Scholar Webinar: Transparent, Open, and Reproducible Research
Digital Scholar Webinar: Transparent, Open, and Reproducible Research
 
Sense About Science: Peer Review Survey
Sense About Science: Peer Review SurveySense About Science: Peer Review Survey
Sense About Science: Peer Review Survey
 
Research Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific MisconductResearch Skills Session 8: Avoid Scientific Misconduct
Research Skills Session 8: Avoid Scientific Misconduct
 

Similar to Towards knowledge maintenance in scientific digital libraries with the keystone framework -jcdl2020--2020-08-02

Mathew.ppt
Mathew.pptMathew.ppt
Mathew.ppt
SurbhiTanwar12
 
Literature review niroj dahal-2017
Literature review niroj dahal-2017Literature review niroj dahal-2017
Literature review niroj dahal-2017
Niroj Dahal
 
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
SPEESUTChapter
 
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
National Information Standards Organization (NISO)
 
British Library
British LibraryBritish Library
British Library
clarivate
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over time
GESIS
 
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
Jamie Bisset
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic Commitments
Simon Knight
 
Systematic literature review technique.pptx
Systematic literature review technique.pptxSystematic literature review technique.pptx
Systematic literature review technique.pptx
TANMAY DAS GUPTA
 
Ab case study assignment help Australia
Ab case study assignment help  AustraliaAb case study assignment help  Australia
Ab case study assignment help Australia
asmits kharel
 
Ab case study assignment help Australia
Ab case study assignment help AustraliaAb case study assignment help Australia
Ab case study assignment help Australia
asmits kharel
 
Citation metrics
Citation metricsCitation metrics
Citation metrics
Vasantha Raju N
 
Scientific research and publication walk through
Scientific research and publication walk throughScientific research and publication walk through
Scientific research and publication walk through
Roshni Mehta
 
research_methodology_presentation_final.pptx
research_methodology_presentation_final.pptxresearch_methodology_presentation_final.pptx
research_methodology_presentation_final.pptx
ran17april2001
 
Scientific Data overview of Data Descriptors - WT Data-Literature integration...
Scientific Data overview of Data Descriptors - WT Data-Literature integration...Scientific Data overview of Data Descriptors - WT Data-Literature integration...
Scientific Data overview of Data Descriptors - WT Data-Literature integration...
Susanna-Assunta Sansone
 
researchmethodologyi-140707092303-phpapp02.pdf
researchmethodologyi-140707092303-phpapp02.pdfresearchmethodologyi-140707092303-phpapp02.pdf
researchmethodologyi-140707092303-phpapp02.pdf
Mdali657802
 
Writing report
Writing reportWriting report
Writing report
Nirmal Singh
 
Research Methodology Part I
Research Methodology Part IResearch Methodology Part I
Research Methodology Part I
Anwar Siddiqui
 
Scientific Data and peer review session at Dryad event, May 2015
Scientific Data and peer review session at Dryad event, May 2015 Scientific Data and peer review session at Dryad event, May 2015
Scientific Data and peer review session at Dryad event, May 2015
Susanna-Assunta Sansone
 
Research Method for Business chapter 3
Research Method for Business chapter 3Research Method for Business chapter 3
Research Method for Business chapter 3
Mazhar Poohlah
 

Similar to Towards knowledge maintenance in scientific digital libraries with the keystone framework -jcdl2020--2020-08-02 (20)

Mathew.ppt
Mathew.pptMathew.ppt
Mathew.ppt
 
Literature review niroj dahal-2017
Literature review niroj dahal-2017Literature review niroj dahal-2017
Literature review niroj dahal-2017
 
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
SPE ESUT TECHNICAL SERIES (ACADEMIC PROJECT WRITING AND PRESENTATION) BY ENGR...
 
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...Warren-Jones "Using text-mining and summarisation technology to manage the gr...
Warren-Jones "Using text-mining and summarisation technology to manage the gr...
 
British Library
British LibraryBritish Library
British Library
 
Highly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over timeHighly cited references in PLOS ONE and their in-text usage over time
Highly cited references in PLOS ONE and their in-text usage over time
 
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
Bibliometrics, Journal Impact Factors and Maximising the Cite-ability of Jour...
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic Commitments
 
Systematic literature review technique.pptx
Systematic literature review technique.pptxSystematic literature review technique.pptx
Systematic literature review technique.pptx
 
Ab case study assignment help Australia
Ab case study assignment help  AustraliaAb case study assignment help  Australia
Ab case study assignment help Australia
 
Ab case study assignment help Australia
Ab case study assignment help AustraliaAb case study assignment help Australia
Ab case study assignment help Australia
 
Citation metrics
Citation metricsCitation metrics
Citation metrics
 
Scientific research and publication walk through
Scientific research and publication walk throughScientific research and publication walk through
Scientific research and publication walk through
 
research_methodology_presentation_final.pptx
research_methodology_presentation_final.pptxresearch_methodology_presentation_final.pptx
research_methodology_presentation_final.pptx
 
Scientific Data overview of Data Descriptors - WT Data-Literature integration...
Scientific Data overview of Data Descriptors - WT Data-Literature integration...Scientific Data overview of Data Descriptors - WT Data-Literature integration...
Scientific Data overview of Data Descriptors - WT Data-Literature integration...
 
researchmethodologyi-140707092303-phpapp02.pdf
researchmethodologyi-140707092303-phpapp02.pdfresearchmethodologyi-140707092303-phpapp02.pdf
researchmethodologyi-140707092303-phpapp02.pdf
 
Writing report
Writing reportWriting report
Writing report
 
Research Methodology Part I
Research Methodology Part IResearch Methodology Part I
Research Methodology Part I
 
Scientific Data and peer review session at Dryad event, May 2015
Scientific Data and peer review session at Dryad event, May 2015 Scientific Data and peer review session at Dryad event, May 2015
Scientific Data and peer review session at Dryad event, May 2015
 
Research Method for Business chapter 3
Research Method for Business chapter 3Research Method for Business chapter 3
Research Method for Business chapter 3
 

More from jodischneider

Continued citation of bad science and what we can do about it--2021-04-20
Continued citation of bad science and what we can do about it--2021-04-20Continued citation of bad science and what we can do about it--2021-04-20
Continued citation of bad science and what we can do about it--2021-04-20
jodischneider
 
Continued citation of bad science and what we can do about it--2021-02-19
Continued citation of bad science and what we can do about it--2021-02-19Continued citation of bad science and what we can do about it--2021-02-19
Continued citation of bad science and what we can do about it--2021-02-19
jodischneider
 
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
jodischneider
 
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
jodischneider
 
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
jodischneider
 
Problem-citations--CrossrefLive18--2018-11-13
Problem-citations--CrossrefLive18--2018-11-13Problem-citations--CrossrefLive18--2018-11-13
Problem-citations--CrossrefLive18--2018-11-13
jodischneider
 
Problematic citations--Workshop-on-Open-Citations--2018-09-03
Problematic citations--Workshop-on-Open-Citations--2018-09-03Problematic citations--Workshop-on-Open-Citations--2018-09-03
Problematic citations--Workshop-on-Open-Citations--2018-09-03
jodischneider
 
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
jodischneider
 
Innovations in reasoning about health: the case of the Randomized Clinical Tr...
Innovations in reasoning about health: the case of the Randomized Clinical Tr...Innovations in reasoning about health: the case of the Randomized Clinical Tr...
Innovations in reasoning about health: the case of the Randomized Clinical Tr...
jodischneider
 
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
jodischneider
 
Rhetorical moves and audience considerations in the discussion sections of ra...
Rhetorical moves and audience considerations in the discussion sections of ra...Rhetorical moves and audience considerations in the discussion sections of ra...
Rhetorical moves and audience considerations in the discussion sections of ra...
jodischneider
 
Citation practices and the construction of scientific fact--ECA-facts-preconf...
Citation practices and the construction of scientific fact--ECA-facts-preconf...Citation practices and the construction of scientific fact--ECA-facts-preconf...
Citation practices and the construction of scientific fact--ECA-facts-preconf...
jodischneider
 
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
jodischneider
 
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
jodischneider
 
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
jodischneider
 
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
jodischneider
 
Persons, documents, models: organising and structuring information for the We...
Persons, documents, models: organising and structuring information for the We...Persons, documents, models: organising and structuring information for the We...
Persons, documents, models: organising and structuring information for the We...
jodischneider
 
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
jodischneider
 
Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23
jodischneider
 
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
jodischneider
 

More from jodischneider (20)

Continued citation of bad science and what we can do about it--2021-04-20
Continued citation of bad science and what we can do about it--2021-04-20Continued citation of bad science and what we can do about it--2021-04-20
Continued citation of bad science and what we can do about it--2021-04-20
 
Continued citation of bad science and what we can do about it--2021-02-19
Continued citation of bad science and what we can do about it--2021-02-19Continued citation of bad science and what we can do about it--2021-02-19
Continued citation of bad science and what we can do about it--2021-02-19
 
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...
 
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02
 
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...
 
Problem-citations--CrossrefLive18--2018-11-13
Problem-citations--CrossrefLive18--2018-11-13Problem-citations--CrossrefLive18--2018-11-13
Problem-citations--CrossrefLive18--2018-11-13
 
Problematic citations--Workshop-on-Open-Citations--2018-09-03
Problematic citations--Workshop-on-Open-Citations--2018-09-03Problematic citations--Workshop-on-Open-Citations--2018-09-03
Problematic citations--Workshop-on-Open-Citations--2018-09-03
 
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...
 
Innovations in reasoning about health: the case of the Randomized Clinical Tr...
Innovations in reasoning about health: the case of the Randomized Clinical Tr...Innovations in reasoning about health: the case of the Randomized Clinical Tr...
Innovations in reasoning about health: the case of the Randomized Clinical Tr...
 
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
Viewing universities as landscapes of scholarship, VIVO keynote, 2017-08-04
 
Rhetorical moves and audience considerations in the discussion sections of ra...
Rhetorical moves and audience considerations in the discussion sections of ra...Rhetorical moves and audience considerations in the discussion sections of ra...
Rhetorical moves and audience considerations in the discussion sections of ra...
 
Citation practices and the construction of scientific fact--ECA-facts-preconf...
Citation practices and the construction of scientific fact--ECA-facts-preconf...Citation practices and the construction of scientific fact--ECA-facts-preconf...
Citation practices and the construction of scientific fact--ECA-facts-preconf...
 
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...
 
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...
 
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...
 
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
Acquiring and representing drug-drug interaction knowledge and evidence, TRIA...
 
Persons, documents, models: organising and structuring information for the We...
Persons, documents, models: organising and structuring information for the We...Persons, documents, models: organising and structuring information for the We...
Persons, documents, models: organising and structuring information for the We...
 
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
Synthesizing knowledge from disagreement -- Manchester -- 2015-05-06
 
Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23Synthesizing knowledge from disagreement -cwi-2015-04-23
Synthesizing knowledge from disagreement -cwi-2015-04-23
 
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
Packaging ideas--nanopublications-in-the-humanities--Europeana--2015-04-21
 

Recently uploaded

GBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of ProteinsGBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
Areesha Ahmad
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
PsychoTech Services
 
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills MN
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
hozt8xgk
 
molar-distalization in orthodontics-seminar.pptx
molar-distalization in orthodontics-seminar.pptxmolar-distalization in orthodontics-seminar.pptx
molar-distalization in orthodontics-seminar.pptx
Anagha Prasad
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
University of Hertfordshire
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
Modelo de slide quimica para powerpoint
Modelo  de slide quimica para powerpointModelo  de slide quimica para powerpoint
Modelo de slide quimica para powerpoint
Karen593256
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
PirithiRaju
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
Leonel Morgado
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
vluwdy49
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
by6843629
 
Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.
Aditi Bajpai
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
PirithiRaju
 
aziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobelaziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobel
İsa Badur
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Leonel Morgado
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
Leonel Morgado
 
Gadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdfGadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdf
PirithiRaju
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
Frédéric Baudron
 
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdfwaterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
LengamoLAppostilic
 

Recently uploaded (20)

GBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of ProteinsGBSN - Biochemistry (Unit 6) Chemistry of Proteins
GBSN - Biochemistry (Unit 6) Chemistry of Proteins
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
 
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
Travis Hills of MN is Making Clean Water Accessible to All Through High Flux ...
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
 
molar-distalization in orthodontics-seminar.pptx
molar-distalization in orthodontics-seminar.pptxmolar-distalization in orthodontics-seminar.pptx
molar-distalization in orthodontics-seminar.pptx
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
Modelo de slide quimica para powerpoint
Modelo  de slide quimica para powerpointModelo  de slide quimica para powerpoint
Modelo de slide quimica para powerpoint
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
 
Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.Micronuclei test.M.sc.zoology.fisheries.
Micronuclei test.M.sc.zoology.fisheries.
 
Pests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdfPests of Storage_Identification_Dr.UPR.pdf
Pests of Storage_Identification_Dr.UPR.pdf
 
aziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobelaziz sancar nobel prize winner: from mardin to nobel
aziz sancar nobel prize winner: from mardin to nobel
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
 
Gadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdfGadgets for management of stored product pests_Dr.UPR.pdf
Gadgets for management of stored product pests_Dr.UPR.pdf
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
 
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdfwaterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
waterlessdyeingtechnolgyusing carbon dioxide chemicalspdf
 

Towards knowledge maintenance in scientific digital libraries with the keystone framework -jcdl2020--2020-08-02

  • 1. Towards Knowledge Maintenance in Scientific Digital Libraries with the Keystone Framework Yuanxi Fu & Jodi Schneider School of Information Sciences University of Illinois at Urbana-Champaign Presentation for JCDL 2020, Virtual, 2020-08-02
  • 2. Motivation: Need for KNOWLEDGE MAINTENANCE • When you’re not an expert, how do you judge papers? – Recency – Citation count – Other heuristics • Literature becomes obsolete but usually that’s not explicit. • These errors can be passed on & can lead to misinterpretations. – Most research needs information from cognate fields.
  • 3. How big is retraction & citation to retraction • Over 600,000 articles directly cite a retracted paper. • The Retraction Watch Database lists over 19,000 retracted publications. • In biomedicine 94% of retracted papers have received at least one citation, with an average citation count of 35 (Dinh, …, Schneider 2019).
  • 4. Motivating Questions • Are papers citing a retracted paper necessarily wrong? • Does it matter when citing authors make use of a paper whose findings are no longer considered valid? • When DOES the citation matter? • Could we selectively alert authors who cite a retracted or abandoned paper?
  • 6. Under our framework: 1) A scientific research paper puts forward at least one main finding, along with a logical argument, giving reasons and evidence to support the main finding. 2) The main finding is accepted (or not) on the basis of the logical argument. 3) Evidence from earlier literature may be incorporated into the argument by citing a paper and presenting it as support, using a citation context.
  • 7. Main Finding Arguments support support Data Methods Citations … Applying the Framework Step 1, model the argument
  • 8. Citing ArticleCited Article Citation Context “Many papers with known validity problems are highly cited [3].” Our paper [3] = Bar-Ilan, J. and Halevi, G. 2018. Temporal characteristics of retracted articles Applying the Framework: Step 2, find the citation contexts that contribute to the argument
  • 9. Applying the Framework: Step 3, analyze the citation context: How many items are cited? Citation Context “Many papers with known validity problems are highly cited [3].” Singleton Cluster “Digital library applications of argumentation theory include argument-based retrieval [21, 30]”
  • 10. Cited Article Citation Context “Many papers with known validity problems are highly cited [3].” [3] = Bar-Ilan, J. and Halevi, G. 2018. Temporal characteristics of retracted articles Applying the Framework: Step 4, analyze the cited article: What kind of support does it give? Main Findings Support
  • 11. Applying the Keystone Framework to Knowledge Maintenance Tasks
  • 12. Case Study 1: Citing Non-reproducible Code Assess the real impact of citing an unreliable paper
  • 13. Case Study 2: Citations Supporting One Paper’s Argument Curate a high impact paper (citation count > 1000) to find its keystone citations (de Calignon et al., 2012)
  • 14. Experts & Non-experts Step 4 Flag those articles that are potentially impacted. Workflow for Assessing the Citations of an Unreliable Paper Step 2 The domain expert develops a list of screening questions. Step 3 Experts/non- experts/text mining tools screen target articles using the check list. Step 1 The domain expert develops a generalized argument model. Experts
  • 15. Workflow for Targeted Curation of Important Papers Step 1 Identify main findings of the article from abstract, conclusion section, and section titles. Step 2 Construct an argument diagram for each main finding. Experts Step 3 Align citations and citation contexts to components in the argument diagrams. Step 4 Identify and categorize keystone citation contexts and keystone citations.
  • 16. Results from Case Study 1 Our assessment # of papers Why? What to do? Unaffected by the code glitch 6 Didn’t directly use the protocol. Cited it to either support decisions they made in their calculations or use it as background information. Nothing! Affected by the code glitch 4 Followed the protocol. Calculations supported claims that went into abstracts or the conclusion section. Authors of the 4 potentially affected papers should double-check their results and either amend their claims or document how the claims are sustained despite the code glitch.
  • 17. Results from Case Study 2 • 51 citations in the whole paper • 5 singleton keystone citations, including • 3 main finding singleton keystone citations support the choice of experimental materials • 1 pass-through keystone citation support the choice of experimental materials • 1 singleton main finding keystone citation support the interpretation of data • One keystone citation cluster with 3 reference items supports the choice of experimental method.
  • 18. An ad-hoc literature review from pass-through keystone citation 1 was used to support the choice of PSD-95 as a synaptic marker.
  • 19. Research Agenda • Scale up: Large-scale identification of keystone citations – Test argument-based curation approaches that have already been automated (e.g. rhetorical-based approaches, Teufel & Kan, 2009) – Develop text mining tools to aid manual curation and screening • Understand citation behaviors, esp. pass-through citations • Develop a taxonomy of validity for indicating the confidence of a reader can have in relying on or reusing the methods and findings of a paper
  • 20. References Bar-Ilan, J. and Halevi, G. 2018. Temporal characteristics of retracted articles. Scientometrics. 116, 3 (Jun. 2018), 1771–1783. https://doi.org/10.1007/s11192-018-2802-y Clark, T., Ciccarese, P., & Goble, C.A. (2014). Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. Journal of Biomedical Semantics, 5, 28. https://doi.org/10.1186/2041-1480-5-28 Dinh, L., Sarol, J., Cheng, Y., Hsiao, T., Parulian, N.N., & Schneider, J. (2019). Systematic Examination of Pre- and Post-Retraction Citations. Proceedings of ASIST, 56(1), 390-394 https://doi.org/10.1002/pra2.35 Green, N.L. (2017). Argumentation Mining in Scientific Discourse. CMNA@ICAIL. http://ceur-ws.org/Vol-2048/paper02.pdf Greenberg, S.A. (2009). How citation distortions create unfounded authority: analysis of a citation network. The BMJ, 339, b2680. https://doi.org/10.1136/bmj.b2680 Teufel, S., & Kan, M. (2009). Robust Argumentative Zoning for Sensemaking in Scholarly Documents. NLP4DL/AT4DL, 154-170. https://doi.org/10.1007/978-3-642-23160-5_10 de Calignon, A., Polydoro, M., Suárez-Calvet, M., William, C., Adamowicz, D. H., Kopeikina, K. J., Pitstick, R., Sahara, N., Ashe, K. H., Carlson, G. A., Spires-Jones, T. L., & Hyman, B. T. (2012). Propagation of tau pathology in a model of early Alzheimer's disease. Neuron, 73(4), 685–697. https://doi.org/10.1016/j.neuron.2011.11.033
  • 22. Appendix A Keystone Citations Found in Case Study 2
  • 23. Three Types of Keystone Citation Contexts Observed Properties of the keystone citation context Removing the citation context would weaken the argument supporting a main finding Only one paper is cited. The main findings of the cited paper(s) provide evidence to support the argument. Corresponding cited article Singleton, main- findings support + + + Main-finding keystone citation Cluster, main-findings support + - + Main-finding keystone citation cluster Singleton, pass- through support + + - Pass-through keystone citation
  • 24. Three main-finding keystone citations supported the choice of experimental materials. Main-finding keystone citation to support choice of material
  • 25. The fourth main-finding keystone citation supports the interpretation of experimental data. Main-finding keystone citation to support data interpretation
  • 26. An ad-hoc literature review from pass-through keystone citation 1 was used to support the choice of PSD-95 as a synaptic marker. Main-finding keystone citation cluster to support material
  • 27. A main-finding keystone citation cluster supports the choice of experimental method. Main-finding keystone citation cluster to support method
  • 29. Concepts in the Framework • Keystone statement: any statement whose unreliability threatens the argument for a main finding of a paper. • Keystone citation context: citation contexts supporting keystone statements. – singleton vs. cluster citation context. • A singleton citation context cites one item, e.g. ‘[2]’ • A cluster citation context cites multiple items, e.g., ‘[2, 16]’ or ‘(DeKosky and Scheff, 1990; Scheff and Price, 2006; Terry et al., 1991)’.
  • 30. Concepts in the Framework • Keystone statement: any statement whose unreliability threatens the argument for a main finding of a paper. • Keystone citation context: citation contexts supporting keystone statements. – Number of supporting items • A singleton citation context cites one item, e.g. ‘[2]’ • A cluster citation context cites multiple items, e.g., ‘[2, 16]’ or ‘(DeKosky and Scheff, 1990; Scheff and Price, 2006; Terry et al., 1991)’.
  • 31. Concepts in the Framework • Keystone statement: any statement whose unreliability threatens the argument for a main finding of a paper. • Keystone citation context: citation contexts supporting keystone statements. – Strength of support • Main-findings support, if the citation context closely relates to a main finding of the cited item. • Pass-through support, if support can be found within the cited item but only in an unsupported statement or a statement referencing one or more other work(s). • No clear support, if the citation context does not clearly relate to the cited item, either its main findings, or other statements it makes.
  • 32. Concepts in the Framework • Keystone Statement (KS): any statement whose unreliability threatens the argument for a main finding of a paper. Main Finding Tau pathology results in neurodegeneration Data support Method Synaptic marker support Keystone Statement XXX is a good synaptic biomarker for detecting neurodegeneration
  • 33. Concepts in the Framework • Keystone Citation Context (KCC): citation contexts that supports keystone statements. Keystone Statement XXX is a good synaptic biomarker for detecting neurodegeneration Keystone Citation Context support
  • 34. Keystone Citation Contexts Continued How many items are cited? Singleton Cluster Whether the cited item’s main findings support the citation context? Main-finding support Pass-through support No support
  • 35. Distinction between KS and KCC • KSs are summation of KCCs. Domain experts can distill the same KS from several KCCs found in different papers. • One study found that several citation contexts can be distilled to a statement about β amyloid accumulation, but the cited papers mentioned the statement only as “hypothesis” or not at all. Greenberg, 2009 Statement: The accumulation of β amyloid occurs early and precedes other abnormalities. Citing Article Cited Article The appearance of Aβ- positive, noncongophilic deposits precedes vacuolization in IBM muscle fibers.8 (as fact) Some muscle fibers had Aβ-positive accumulations,… Those muscle fibers,…, may represent early changes of IBM. (as hypothesis) support
  • 37. (Teufel & Kan, 2011) Rhetoric-based Approaches • Extract information based on rhetorical feature • Limited need for domain knowledge • Provide a coarser argumentative structure • Well-automated
  • 38. Argument-scheme based Approaches Premises: • A group of individuals G have atypical phenotype P • All of the individuals in G have atypical genotype M • Another group of individuals (controls) do not have P • None of controls have M Conclusion: M may be the cause of P (in G) (Green, 2017) • Require deep domain analysis • Show the logic of how a discipline justifies its findings • Identify potential weakness through critical questions • Currently depend on manual curation, but potentially automatable through text mining
  • 39. Provenance-based Method (Clark et al., 2014) • Model scientists’ work process • Most suitable for modeling empirical research articles • Require manual curation