Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base--LISC2014--2014-10-19
This document discusses a project to construct a knowledge base linking drug interaction assertions to evidence from source documents. It will use the Micropublications Ontology to represent each assertion's support graph of claims and evidence, and the Open Annotation model to dynamically link support graph elements to quoted text excerpts from sources. The knowledge base will help answer competency questions about assertions, evidence, and their provenance. Challenges include representing both structured and unstructured text claims and efficiently querying the evidence base at scale.
SocialCite makes its debut at the HighWire Press meetingKent Anderson
A new service designed to allow readers and researchers to comment on the appropriateness, quality, and type of citations made in the literature made its debut at the HighWire Press Publishers Meeting yesterday.
Annotation examples. This is an overview of some of the software I have used for annotation (and a few extra features some of this software has.) This was presented in the SwissUniversities Doctoral Programme, Language & Cognition, in the Module: Linguistic and corpus perspectives on argumentative discourse.
Screenshots are given of GATE, UAM Corpus Tool, Excel, BRAT, EPPI Reviewer, and a custom tool. In most cases there are references to one of my papers for further details.
I briefly describe a typical annotation process:
Find text of interest
Find phenomena of interest
Draft an annotation manual
Iteratively test annotation & revise manual
Find questionable annotations, check disagreements.
Revise the manual.
Iterate.
Annotate
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...jodischneider
Presentation to the INRIA WIMMICS research group 2014-10-17 about our LISC paper: Using the micropublication ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base:
http://jodischneider.com/pubs/lisc2014.pdf
http://wimmics.inria.fr/seminars
II-SDV 2016 Srinivasan Parthiban - KOL Analytics from Biomedical LiteratureDr. Haxel Consult
Strategic partnerships between pharmaceutical companies and medical experts lead to more effective medical and marketing activities throughout a product life cycle. Identification of such medical experts, that is, key opinion leaders (KOLs) from bibliometric analysis is challenging due to volume and variety of data. Today, the research community is flooded with scientific literature, with thousands of journals and over 20 million abstracts in PubMed. Developing a holistic framework to identify, profile and update the KOLs is the need of the hour. Customers want digestible information – everything relevant. In this talk, we will present case studies on how we used the ontologies and disambiguation techniques to address KOL identification for different therapeutic areas.
Do Open data badges influence author behaviour? A case study at Springer NatureRebecca Grant
Digital badges have previously been shown to incentivise journal authors to share their data openly. In this paper we introduce an Open data badging project at the Springer Nature journal BMC Microbiology. The development of the Open data badge is described, as well as the challenges of developing standard badging criteria and ensuring authors’ awareness of the badges. Next steps for the badging project are outlined, which are based on the experiences of the team assessing the badges, the number of badges awarded at the journal to date, and the results of an author survey.
Research in the time of Covid: Surveying impacts on Early Career ResearchersRebecca Grant
Based on a survey of over 4,500 researchers published in the white paper The State of Open Data 2020, this session will explore the impacts of the pandemic on early career reearchers (ECRs), their research practice, and how they interact with open data. We will discuss the specific challenges reported by ECRs, as well as the gaps in training and support that they have identified that would encourage their sharing and reuse of research data.
Presentation at the E-ARMA conference 2021.
Increasing transparency in Medical Education through Open Data Rebecca Grant
Slides presented at the AMEE Virtual Conference 2021, introducing the MedEdPublish platform and data policies. Approaches to sharing sensitive human data, and particulary qualitative data, are discussed.
SocialCite makes its debut at the HighWire Press meetingKent Anderson
A new service designed to allow readers and researchers to comment on the appropriateness, quality, and type of citations made in the literature made its debut at the HighWire Press Publishers Meeting yesterday.
Annotation examples. This is an overview of some of the software I have used for annotation (and a few extra features some of this software has.) This was presented in the SwissUniversities Doctoral Programme, Language & Cognition, in the Module: Linguistic and corpus perspectives on argumentative discourse.
Screenshots are given of GATE, UAM Corpus Tool, Excel, BRAT, EPPI Reviewer, and a custom tool. In most cases there are references to one of my papers for further details.
I briefly describe a typical annotation process:
Find text of interest
Find phenomena of interest
Draft an annotation manual
Iteratively test annotation & revise manual
Find questionable annotations, check disagreements.
Revise the manual.
Iterate.
Annotate
Wimmics seminar--drug interaction knowledge base, micropublication, open anno...jodischneider
Presentation to the INRIA WIMMICS research group 2014-10-17 about our LISC paper: Using the micropublication ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base:
http://jodischneider.com/pubs/lisc2014.pdf
http://wimmics.inria.fr/seminars
II-SDV 2016 Srinivasan Parthiban - KOL Analytics from Biomedical LiteratureDr. Haxel Consult
Strategic partnerships between pharmaceutical companies and medical experts lead to more effective medical and marketing activities throughout a product life cycle. Identification of such medical experts, that is, key opinion leaders (KOLs) from bibliometric analysis is challenging due to volume and variety of data. Today, the research community is flooded with scientific literature, with thousands of journals and over 20 million abstracts in PubMed. Developing a holistic framework to identify, profile and update the KOLs is the need of the hour. Customers want digestible information – everything relevant. In this talk, we will present case studies on how we used the ontologies and disambiguation techniques to address KOL identification for different therapeutic areas.
Do Open data badges influence author behaviour? A case study at Springer NatureRebecca Grant
Digital badges have previously been shown to incentivise journal authors to share their data openly. In this paper we introduce an Open data badging project at the Springer Nature journal BMC Microbiology. The development of the Open data badge is described, as well as the challenges of developing standard badging criteria and ensuring authors’ awareness of the badges. Next steps for the badging project are outlined, which are based on the experiences of the team assessing the badges, the number of badges awarded at the journal to date, and the results of an author survey.
Research in the time of Covid: Surveying impacts on Early Career ResearchersRebecca Grant
Based on a survey of over 4,500 researchers published in the white paper The State of Open Data 2020, this session will explore the impacts of the pandemic on early career reearchers (ECRs), their research practice, and how they interact with open data. We will discuss the specific challenges reported by ECRs, as well as the gaps in training and support that they have identified that would encourage their sharing and reuse of research data.
Presentation at the E-ARMA conference 2021.
Increasing transparency in Medical Education through Open Data Rebecca Grant
Slides presented at the AMEE Virtual Conference 2021, introducing the MedEdPublish platform and data policies. Approaches to sharing sensitive human data, and particulary qualitative data, are discussed.
Software Repositories for Research -- An Environmental ScanMicah Altman
Presented at the Software Preservation Network Forum:
"We discuss the results of an environmental scan characterizing the current landscape of software repositories, hubs, and publication venues that are used in research and scholarships. The study aims to characterize the research and scholarship use cases supported by exemplar repositories, their models for sustainability, and the related key affordances, significant properties which the repository offers/maintains. We supplement this with a scan of funder and publisher policies toward software curation and citation; and a summary of key policy resources and guidelines. Using this environmental scan, we discuss a preliminary gap analysis. It hoped that by addressing these key questions, new insights will be provided into the types of decisions research Libraries can expect to make when designing future pilot software curation services."
Digital Scholar Webinar: Recruiting Research Participants Online Using RedditSC CTSI at USC and CHLA
This 50-minute presentation introduces r/SampleSize, a community on the website Reddit that allows for online participant recruitment without compulsory or immediate payment. It will provide an overview of best practices for recruiting participants on r/SampleSize. It will also compare r/SampleSize to Amazon Mechanical Turk (MTurk), a widely used crowdsourcing platform for recruiting research participants.
This presentation was provided by Emma Warren-Jones of Scholarcy, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
Research Data Sharing and Re-Use: Practical Implications for Data Citation Pr...SC CTSI at USC and CHLA
Date: Apr 4, 2018
Speaker: Hyoungjoo Park, PhD candidate, School of Information Studies, University of Wisconsin-Milwaukee, and Dietmar Wolfram, PhD
Overview: It is increasingly common for researchers to make their data freely available. This is often a requirement of funding agencies but also consistent with the principles of open science, according to which all research data should be shared and made available for reuse. Once data is reused, the researchers who have provided access to it should be acknowledged for their contributions, much as authors are recognised for their publications through citation. Hyoungjoo Park and Dietmar Wolfram have studied characteristics of data sharing, reuse, and citation and found that current data citation practices do not yet benefit data sharers, with little or no consistency in their format. More formalised citation practices might encourage more authors to make their data available for reuse.
Journal Club - Best Practices for Scientific ComputingBram Zandbelt
Journal Club presentation for Cools lab at Donders Institute, Radboud University, Nijmegen, the Netherlands
Date: October 28, 2015
Paper:
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., ... & Wilson, P. (2014). Best practices for scientific computing. PLoS Biology, 12(1), e1001745.
Open science and the individual researcherBram Zandbelt
Slides for the Feb 8, 2017 lab meeting of Roshan Cools' Motivation & Cognitive Control group (Donders Institute), discussing the following paper:
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., … Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800. https://doi.org/10.7554/eLife.16800.
This presentation was provided by Stephanie Roth of Temple University, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
This presentation was provided by Bert Carelli of TrendMD, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
This discussion, covened by the Dubai Future Foundation, focusses on identifying the significance of the concept of well-being for social-science and policy; and the opportunities to measure it at scale.
Pistoia Alliance Debates: Text Mining for Pharma R&D in a Social World (17th ...Pistoia Alliance
Text-mining of journal articles and other publications has long been a subject of interest. It already has applications across R&D and beyond into health care, for instance by analysing electronic health records. The technology has value but also has its limits. With new sources of text to mine becoming mainstream, such as Twitter feeds or Facebook posts that might reference a company’s brand or a drug’s efficacy or adverse events, existing technology needs to be adapted to keep pace. Not only that, but whole new compliance questions arise: does a fleeting mention on Twitter require the same response as a formal notification of an adverse event?
The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson,” captured the spotlight and stimulated the imagination of the entire world. The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created a high level of interest in the healthcare community about the potential of this breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine. Dr. Siegel is currently working together with IBM engineers to explore how Dr. Watson can work together with physicians and medical specialists. His presentation, which was delivered on March 28th, provided a high level overview of the uniqueness of Deep Q/A Software and how it differs from other previous artificial intelligence applications.
A poster presented at the 2016 Annual Meeting of the Medical Library Association on a strategy for identifying emerging technologies through Pubmed searching. This is an outcome from the MLA systematic review project from the association's research initiative.
Building a Network of Interoperable and Independently Produced Linked and Ope...Michel Dumontier
Over 15 years ago, Sir Tim Berners Lee proclaimed the founding of an exciting new future involving intelligent agents operating over smarter data in order to perform complex tasks at the behest of their human controllers. At the heart of this vision lies an uneasy alliance between tedious formal knowledge representations and powerful analytics over big, but often messy data. Bio2RDF, our decade old open source project to create Linked Data for the life sciences, has weaved emergent Semantic Web technologies such as ontologies and Linked Data to generate FAIR - Findable, Accessible, Interoperable, and Reusable - data in the form of billions of machine accessible statements for use in downstream biomedical discovery.
This revolution in data publication has been strengthened by action from global bioinformatics institutions such as the NCBI, NCBO, EBI, and DBCLS. Notably, NCBI's PubChem has successfully coupled large scale data integration with community-based standards to offer a remakable biochemical knowledge resource amenable to data hungry discovery tools. Yet, in the face of increasing pressure from researchers, funders, and publishers, will these approaches be sufficient for growing and maintaining a comprehensive knowledge graph that is inclusive of all biomedical research?
Software Repositories for Research -- An Environmental ScanMicah Altman
Presented at the Software Preservation Network Forum:
"We discuss the results of an environmental scan characterizing the current landscape of software repositories, hubs, and publication venues that are used in research and scholarships. The study aims to characterize the research and scholarship use cases supported by exemplar repositories, their models for sustainability, and the related key affordances, significant properties which the repository offers/maintains. We supplement this with a scan of funder and publisher policies toward software curation and citation; and a summary of key policy resources and guidelines. Using this environmental scan, we discuss a preliminary gap analysis. It hoped that by addressing these key questions, new insights will be provided into the types of decisions research Libraries can expect to make when designing future pilot software curation services."
Digital Scholar Webinar: Recruiting Research Participants Online Using RedditSC CTSI at USC and CHLA
This 50-minute presentation introduces r/SampleSize, a community on the website Reddit that allows for online participant recruitment without compulsory or immediate payment. It will provide an overview of best practices for recruiting participants on r/SampleSize. It will also compare r/SampleSize to Amazon Mechanical Turk (MTurk), a widely used crowdsourcing platform for recruiting research participants.
This presentation was provided by Emma Warren-Jones of Scholarcy, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
Research Data Sharing and Re-Use: Practical Implications for Data Citation Pr...SC CTSI at USC and CHLA
Date: Apr 4, 2018
Speaker: Hyoungjoo Park, PhD candidate, School of Information Studies, University of Wisconsin-Milwaukee, and Dietmar Wolfram, PhD
Overview: It is increasingly common for researchers to make their data freely available. This is often a requirement of funding agencies but also consistent with the principles of open science, according to which all research data should be shared and made available for reuse. Once data is reused, the researchers who have provided access to it should be acknowledged for their contributions, much as authors are recognised for their publications through citation. Hyoungjoo Park and Dietmar Wolfram have studied characteristics of data sharing, reuse, and citation and found that current data citation practices do not yet benefit data sharers, with little or no consistency in their format. More formalised citation practices might encourage more authors to make their data available for reuse.
Journal Club - Best Practices for Scientific ComputingBram Zandbelt
Journal Club presentation for Cools lab at Donders Institute, Radboud University, Nijmegen, the Netherlands
Date: October 28, 2015
Paper:
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., ... & Wilson, P. (2014). Best practices for scientific computing. PLoS Biology, 12(1), e1001745.
Open science and the individual researcherBram Zandbelt
Slides for the Feb 8, 2017 lab meeting of Roshan Cools' Motivation & Cognitive Control group (Donders Institute), discussing the following paper:
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., … Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800. https://doi.org/10.7554/eLife.16800.
This presentation was provided by Stephanie Roth of Temple University, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
This presentation was provided by Bert Carelli of TrendMD, during the NISO event "Researcher Behaviors and the Impact of Technology," held on March 25, 2020.
This discussion, covened by the Dubai Future Foundation, focusses on identifying the significance of the concept of well-being for social-science and policy; and the opportunities to measure it at scale.
Pistoia Alliance Debates: Text Mining for Pharma R&D in a Social World (17th ...Pistoia Alliance
Text-mining of journal articles and other publications has long been a subject of interest. It already has applications across R&D and beyond into health care, for instance by analysing electronic health records. The technology has value but also has its limits. With new sources of text to mine becoming mainstream, such as Twitter feeds or Facebook posts that might reference a company’s brand or a drug’s efficacy or adverse events, existing technology needs to be adapted to keep pace. Not only that, but whole new compliance questions arise: does a fleeting mention on Twitter require the same response as a formal notification of an adverse event?
The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson,” captured the spotlight and stimulated the imagination of the entire world. The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created a high level of interest in the healthcare community about the potential of this breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine. Dr. Siegel is currently working together with IBM engineers to explore how Dr. Watson can work together with physicians and medical specialists. His presentation, which was delivered on March 28th, provided a high level overview of the uniqueness of Deep Q/A Software and how it differs from other previous artificial intelligence applications.
A poster presented at the 2016 Annual Meeting of the Medical Library Association on a strategy for identifying emerging technologies through Pubmed searching. This is an outcome from the MLA systematic review project from the association's research initiative.
Similar to Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base--LISC2014--2014-10-19
Building a Network of Interoperable and Independently Produced Linked and Ope...Michel Dumontier
Over 15 years ago, Sir Tim Berners Lee proclaimed the founding of an exciting new future involving intelligent agents operating over smarter data in order to perform complex tasks at the behest of their human controllers. At the heart of this vision lies an uneasy alliance between tedious formal knowledge representations and powerful analytics over big, but often messy data. Bio2RDF, our decade old open source project to create Linked Data for the life sciences, has weaved emergent Semantic Web technologies such as ontologies and Linked Data to generate FAIR - Findable, Accessible, Interoperable, and Reusable - data in the form of billions of machine accessible statements for use in downstream biomedical discovery.
This revolution in data publication has been strengthened by action from global bioinformatics institutions such as the NCBI, NCBO, EBI, and DBCLS. Notably, NCBI's PubChem has successfully coupled large scale data integration with community-based standards to offer a remakable biochemical knowledge resource amenable to data hungry discovery tools. Yet, in the face of increasing pressure from researchers, funders, and publishers, will these approaches be sufficient for growing and maintaining a comprehensive knowledge graph that is inclusive of all biomedical research?
Paper was presented at European Survey Research Association 2013, in the session Research Data Management for Re-use: Bringing Researchers and Archivists closer.
Toward F.A.I.R. Pharma. PhUSE Linked Data Initiatives Past and PresentTim Williams
Abstract:
In recent years, the PhUSE organization has supported several Linked Data initiatives. The CDISC Foundational Standards as RDF is an early example of one such initiative. The results are available on the CDISC website. Subsequent proof of concept projects enjoyed marginal success at a time when pharma’s familiarity with the technology was still very limited. A recent surge in interest in F.A.I.R. data and Knowledge Graphs has sparked renewed interest in Linked Data within PhUSE and the industry at large. The recently completed “Clinical Trials Data as RDF (CTDasRDF)” spawned a new project, “Going Translational With Linked Data (GoTWLD).” GoTWLD extends the project scope of its predecessor beyond SDTM into the non-clinical domain.
Educational initiatives at PhUSE include an introductory, interactive workshop at the annual European conference (EU-Connect) and at the US Computational Science Symposium (CSS). A side-project of GoTWLD is investigating the potential use of URIs as study identifiers to promote adoption of Linked Data. Challenges remain, including the need for demonstrable return on investment and the development of user-friendly, intuitive interfaces for graph data. These challenges can be overcome if pharmaceutical companies cooperate in the pre-competitive space.
Presented at Semantics@Roche, Basel 2019-04-04
Lecture for a course at NTNU, 27th January 2021
CC-BY 4.0 Dag Endresen https://orcid.org/0000-0002-2352-5497
See also http://bit.ly/biodiversityinformatics
https://www.gbif.no/events/2021/lecture-ntnu-gbif.html
SciDataCon 2014 Data Papers and their applications workshop - NPG Scientific ...Susanna-Assunta Sansone
Part of the SciDataCon14 workshop on "Data Papers and their applications" run by myself and Brian Hole to help attendees understand current data-publishing journals and trends and help them understand the editorial processes on NPG's Scientific Data and Ubiquity's Open Health Data.
Presentation to the NIAID Office of Cyber Infrastructure and Computational Biology May 5, 2014
Similar to Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base--LISC2014--2014-10-19 (20)
Continued citation of bad science and what we can do about it--2021-04-20jodischneider
Continued Citation of Bad Science and What We Can Do About It
Even papers that falsify data continue to be cited. I describe network and text analysis for studying how authors continue to cite bad science: articles retracted from the literature due to serious flaws or errors. I will present an in-depth case study of a human trial cited for over 10 years after it was retracted for falsifying data. Then, I will describe how the team scaled up to study a data set of 7000 retracted papers and hundreds of thousands of citations. Finally, I will discuss an ongoing Sloan-funded stakeholder consultation that is bringing editors, publishers, librarians, researchers, and research integrity experts together to address this problem.
BiographyJodi Schneider is Assistant Professor at the School of Information Sciences, University of Illinois at Urbana-Champaign where she runs the Information Quality Lab. She studies the science of science through the lens of arguments, evidence, and persuasion with a special interest in controversies in science. Her recent work has focused on topics such as systematic review automation, semantic publication, and the citation of retracted papers. Interdisciplinarity (PhD in Informatics, MS Library & Information Science, MA Mathematics; BA Great Books/liberal arts) is a fundamental principle of her work. She has held research positions across the U.S. as well as in Ireland, England, France, and Chile. She leads the Alfred P. Sloan-funded project, Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda. With Aaron Cohen and Neil Smalheiser she is working on the NIH R01 "Text Mining Pipeline to Accelerate Systematic Reviews in Evidence-Based Medicine". Talk with her about scoping reviews and about citation-based methods for updating systematic reviews!
Tuesday, April 20th, 2021
Noon-1PM Eastern
GWU - CNHS Informatics Seminar
Continued citation of bad science and what we can do about it--2021-02-19jodischneider
Title: Continued Citation of Bad Science and What We Can Do About It
Abstract: Even papers that falsify data continue to be cited. I describe network and text analysis for studying how authors continue to cite bad science: articles retracted from the literature due to serious flaws or errors. Jodi will present an in-depth case study of a human trial cited for over 10 years after it was retracted for falsifying data. Then, will describe how the team scaled up to study a data set of 7000 retracted papers and hundreds of thousands of citations. Finally, Jodi will discuss an ongoing Sloan-funded stakeholder consultation that is bringing editors, publishers, librarians, researchers, and research integrity experts together to address this problem.
The problems of post retraction citation - and mitigation strategies that wor...jodischneider
Presentation for the Bibliometrics & Research Assessment Symposium 2020 (bibSymp20) https://www.nihlibrary.nih.gov/services/bibliometrics/bibSymp20
October 9, 2020
Retraction is intended to remove articles from the citable literature. However, a series of studies from over 30 years, from 1990 through 2020, have found that many retracted papers continue to be cited, and cited positively, even following misconduct-related retractions. For instance, a fraudulent clinical trial report retracted in 2008 continues to receive citations in 2020, and 96% of post-retraction citations do not mention its citation - perhaps because its retraction not marked on the publisher website and its retraction notice cannot be readily retrieved from 7 out of 8 databases (8 out of 9 database records) we tested. This talk draws an ongoing systematic mapping study of research about retraction and our own research projects to summarize what is known about post-retraction citation in biomedicine. We outline practical steps that authors and reviewers can take to avoid being caught out by poorly marked retracted papers.
20 minutes including Q&A
Towards knowledge maintenance in scientific digital libraries with the keysto...jodischneider
JCDL2020 full paper.
Abstract:
Scientific digital libraries speed dissemination of scientific publications, but also the propagation of invalid or unreliable knowledge. Although many papers with known validity problems are highly cited, no auditing process is currently available to determine whether a citing paper’s findings fundamentally depend on invalid or unreliable knowledge. To address this, we introduce a new framework, the keystone framework, designed to identify when and how citing unreliable findings impacts a paper, using argumentation theory and citation context analysis. Through two pilot case studies, we demonstrate how the keystone framework can be applied to knowledge maintenance tasks for digital libraries, including addressing citations of a non-reproducible paper and identifying statements most needing validation in a high-impact paper. We identify roles for librarians, database maintainers, knowledge base curators, and research software engineers in applying the framework to scientific digital libraries.
doi:10.1145/3383583.3398514
Preprint: http://jodischneider.com/pubs/jcdl2020.pdf
Methods Pyramids as an Organizing Structure for Evidence-Based Medicine--SIGC...jodischneider
Keynote talk 2020-08-01 for the JCDL Workshop on Conceptual Models: https://sig-cm.github.io/news/JCDL-2020-CFP/
Discussion points:
* Methods are a key part of the Knowledge Organizing Structure for Evidence-Based Medicine.
* Methods relate to how we GENERATE evidence.
* Different methods generate evidence of different kinds and strength.
* I believe Methods can be useful in mining claims and arguments from papers: methods AUTHORIZE claims.
* More specialized hierarchies of evidence can be found in medicine
* Various groups are complicating the “evidence pyramid” hierarchy of evidence.
Argumentation mining--an introduction for linguists--Fribourg--2019-09-02jodischneider
An introduction to argumentation mining for PhD students. This was presented in the SwissUniversities Doctoral Programme, Language & Cognition, in the Module: Linguistic and corpus perspectives on argumentative discourse. The presentation largely follows Chapters 1-4 and Chapter 10 of my book, Argumentation Mining, co-authored with Manfred Stede in the Synthesis Lectures on Human Language Technologies from Morgan & Claypool: https://doi.org/10.2200/S00883ED1V01Y201811HLT040
Topics:
My book w/computational linguist Manfred Stede: Argumentation Mining
What is argumentation?
Argumentation mining: a first look
Argumentative language
Challenges for argumentation mining
Argumentation structures
Corpus annotation
Why study argumentation mining?
Beyond Randomized Clinical Trials: emerging innovations in reasoning about he...jodischneider
Talk at the 3rd European Conference on Argumentation
ABSTRACT: Specialized fields may at any time invent new inference rules—that is, new warrants—to improve on their stock of resources for drawing and defending conclusions. Yet disagreement over the acceptability of an invented warrant can always be re-opened. Randomized Clinical Trial is widely regarded as the gold standard for making inferences about causal relationships between medical treatments and patient outcomes. Once controversial, RCT achieved broad acceptance within the field as a result of warrant-establishing arguments circulating in the medical literature starting in the 1950s. And RCT has accumulated a very impressive track record of generating new conclusions that withstand critical scrutiny.
Here we look at two emerging innovations whose purpose is to support reasoning about health, offering ways to generate different classes of conclusions. These innovations could be seen as complementary to RCTs, but for both there are also hints of challenge to the enormous prestige of RCTs. We see this most particularly in the gap that has developed between the RCT-generated fact base and the decisions doctors and health policy officials have to make about treatments for patients. We’ve mentioned before that specialized inference methods that become stabilized within an expert community can meet unexpected challenges when they become components of reasoning by other communities. The two innovations considered here each allow us to explore the tensions that arise from the contrasting perspectives of scientists, clinicians, and patients.
Publishers are caretakers of science. Part of that work is maintaining the integrity of scientific literature. Science builds directly upon past work, so we need to be sure that we are building upon a solid foundation and not faulty research. Publishers need to take an active role in monitoring and tracking faulty, retracted research and its influence. I'm asking publishers to (1) clearly mark retracted papers; (2) alert authors who have already cited a retracted paper; and (3) before publishing an article, check its bibliography for retracted papers.
Retracted papers should be clearly marked everywhere they appear, but today that is not the case. Publishers can also use the CrossRef CrossMark service, which lets readers check for article updates (such as retraction) from a little red ribbon at the top of an article. Checking for citations to retracted articles, and limiting future citations, can help science self-correct by shoring up its foundations.
The structure of citation networks provides evidence about how scientific information is diffused. Problematic citation patterns include the selective citation of positive findings, citation bias, as well as the continued citation of retracted literature (i.e. literature formally withdrawn due to error, fraud, or ethical problems). For instance, there is some evidence that positive results tend to receive more citations. The public domain licensing of the Open Citations Corpus makes it possible, in principle, to estimate the likelihood that any network of research papers suffers from problematic citation. To-date, problematic citation been documented ad-hoc, in several striking studies. In Alzheimer's disease research, biased citation, ignoring critical findings, was used to support successful U.S. NIH grant proposals (Greenberg 2009). Mistranslation of obesity research has been used to justify exertion game research (Marshall & Linehan 2017). Citation of fraudulent research about Chronic Obstructive Pulmonary Disease continued after its retraction (Fulton et al. 2015). The data resulting from such studies is of great use to my lab in replicating and determining how to generalize the detection of problematic citation patterns. Previously, the detection of problematic citation patterns has been a side effect of astute researchers, noticing suspicious findings while conducting systematic literature reviews. This talk will describe work-in-progress in my lab detecting problematic citation patterns using natural language processing, combined with network analysis on the Open Citations Corpus.
Modeling Alzheimer’s Disease research claims, evidence, and arguments from a ...jodischneider
Presentation: Jodi Schneider and Novejot Sandhu, “Modeling Alzheimer’s Disease Reseach Claims, Evidence, and Arguments from a Biology Research Paper.” 9th International Conference on Argumentation, International Society for the Society of Argumentation, Amsterdam, Netherlands, July 5, 2018
Abstract: Argument visualization may help make research papers easier to understand, which could both speed quality assessment within a discipline and help build interdisciplinary knowledge networks. This paper presents a case study of the arguments in a single high-profile paper on Alzheimer's disease research. Within this one paper, we analyze and hand-annotate the main claim, which is supported by 4 subclaims, in turn supported by data, methods, and materials. We also investigate how the paper imports and uses knowledge claims from other research papers. We create a specialized argument-based knowledge representation called a micropublication. In future work, we will investigate automatic argumentation mining for experimental biology research papers. Our long-term vision is to create literature-scale claim-argument networks that help more quickly use new knowledge about human health.
Innovations in reasoning about health: the case of the Randomized Clinical Tr...jodischneider
Presentation: Jodi Schneider and Sally Jackson, “Innovations in Reasoning About Health: The Case of the Randomized Clinical Trial.” 9th International Conference on Argumentation, International Society for the Society of Argumentation, Amsterdam, Netherlands, July 5, 2018
Abstract: Field-dependence in argumentation comes about through forms of inference invented by specialized fields. In recent work we introduced the concept of a "warranting device": (1) an inference license (2) invented for a specialized argumentative purpose and (3) backed by institutional, procedural, and material assurances of the dependability of conclusions generated by the device. Once established, fields employ such devices across many situations without further defense, even as the devices develop in response to newly-noticed problems.
Many new warranting devices have appeared over the past century to solve problems in reasoning about health and medicine, replacing and obsolescing earlier forms of medical reasoning. One such device is the Randomized Controlled Trial. This case study traces its historical evolution and discusses some current movements toward competing device types.
Rhetorical moves and audience considerations in the discussion sections of ra...jodischneider
European Conference on Argumentation talk
Jodi Schneider, Graciela Rosemblat, Shabnam Tafreshi and Halil Kilicoglu “Rhetorical moves and audience considerations in the discussion sections of Randomized Controlled Trials of health interventions” [Conference Panel Presentation], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23
1 of 3 talks in Jodi Schneider and Sally Jackson, organizers, “Innovations in Reasoning and Arguing about Health ”[Conference Panel], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23.
Citation practices and the construction of scientific fact--ECA-facts-preconf...jodischneider
Citation practices and the construction of scientific fact. Presentation at the European Conference on Argumentation preconference on status, relevance, and authority of facts.
What WikiCite can learn from biomedical citation networks--Wikicite2017--2017...jodischneider
This is a quick, high-level tour of some ideas from evidence-based medicine, citation-related ontologies for argumentation and evidence curation and biomedicine.
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16...jodischneider
Medication safety as a use case for argumentation mining
We present a use case for argumentation mining, from biomedical informatics, specifically from medication safety. Tens of thousands of preventable medical errors occur in the U.S. each year, due to limitations in the information available to clinicians. Current knowledge sources about potential drug-drug interactions (PDDIs) often fail to provide essential management recommendations and differ significantly in their coverage, accuracy, and agreement. The Drug Interaction Knowledge Base Project (Boyce, 2006-present; dikb.org) is addressing this problem.
Our current work is using knowledge representations and human annotation in order to represent clinically-relevant claims and evidence. Our data model incorporates an existing argumentation-focused ontology, the Micropublications Ontology. Further, to describe more specific information, such as the types of studies that allow inference of a particular type of claim, we are developing an evidence-focused ontology called DIDEO--Drug-drug Interaction and Drug-drug Interaction Evidence Ontology. On the curation side, we will describe how our research team is hand-extracting knowledge claims and evidence from the primary research literature, case reports, and FDA-approved drug labels for 65 drugs.
We think that medication safety could be an important domain for applying automatic argumentation mining in the future. In discussions at Dagstuhl, we would like to investigate how current argumentation mining techniques might be used to scale up this work. We can also discuss possible implications for representing evidence from other biomedical domains.
Talk for Dagstuhl Seminar 16161: Natural Language Argumentation: Mining, Processing, and Reasoning over Textual Arguments
http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=16161
Acquiring and representing drug-drug interaction knowledge and evidence, Litm...jodischneider
Presentation to Diane Litman's lab at the University of Pittsburgh about modeling and acquiring evidence for the Drug Interaction Knowledge Base (DIKB) project.
Persons, documents, models: organising and structuring information for the We...jodischneider
A talk for the Moore Institute for Humanities -
People and documents are of enduring interest. Documents may be generated by individuals, collective groups, and administrations, on any number of topics. We are particularly interested in the relationships between people and documents. The most important relationships are creation (authors, illustrators, translators, ...), usage (e.g. association copies), and topic-of (e.g. people may be the subjects of biographies).
In this lecture, we will talk about several approaches for modeling, or representing, people and documents. We pay particular attention to computer-based approaches to organization, and to organizing information for websites. We will talk briefly about TEI and XML, and the focus on my area of research expertise: modeling "linked data", a widely adopted approach for interlinking data. Adopted by the UK and US governments and search engines such as Google and Yahoo!, linked data has also been widely used in the digital humanities and by libraries, archives, and museums. It consists in naming objects of interest (be they authors, documents, or whatnot) and using standard data formats to enable interlinking.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base--LISC2014--2014-10-19
1. Linked Science at ISWC 2014
Riva del Garda, Trentino, Italy
19 October 2014
Jodi Schneider, Paolo Ciccarese,
Tim Clark and Richard D. Boyce
2. Goal of this project
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
3. Why? It's time-consuming to find
the state of the art in a field!
• What do we know about field F? assertion X?
• What evidence supports assertion X?
• What assumptions are used in research
supporting assertion X?
4. Application domain: medication safety
• Potential drug-drug interactions
– 2+ drugs, where interaction is known to be possible
• Adverse drug event
– Harm caused by medication
– Huge public health issue
> 1.5 million preventable adverse drug events/year
(USA)
• Post-market safety issues
5. Drug information sources
• Evidence is selected & assessed by editorial boards
– MICROMEDEX, First DataBank, Q-DIPS
• E.g. MICROMEDEX:
– "In-house team of 90+ clinically-trained editorial staff"
(physicians, clinical pharmacists, nurses, medical librarians)
– "Content is reviewed for clinical accuracy and relevance."
– "Critical content areas may undergo an additional review by
members of our Editorial Board."
• Potential problems
– a time-consuming (i.e. expensive), collaborative, process
– maintaining internal and external inconsistency is non-trivial
6. Part of a larger effort
• “Addressing gaps in clinically useful evidence on
drug-drug interactions”
• 4-year project, U.S. National Library of Medicine
R01 grant (PI, Richard Boyce)
• Evidence panel of domain experts
(Carol Collins, Lisa Hines, John R Horn, Phil Empey)
& informaticists
(Tim Clark, Paolo Ciccarese, Jodi Schneider)
• Programmer: Yifan Ning
7. Build on 3 things
• Drug Interaction Knowledge Base [Boyce2007,
Boyce2009]
• Open Annotation Data Model [W3C2013]
• Micropublications Ontology [Clark2014]
8. Drug Interaction Knowledge Base
(DIKB)
– Hand-constructed knowledge base
– Safety issues when 2 drugs are taken together
– Focus is on EVIDENCE
[Boyce2007, Boyce2009]
9. Drug Interaction Knowledge Base
(DIKB) - Boyce 2007-2009
– Hand-constructed knowledge base
– Safety issues when 2 drugs are taken together
– Focus is on EVIDENCE
All assumptions are linked to evidence
Enables the [Boyce2007, system Boyce2009]
to identify when
assumptions are no longer valid
10. DIKB supports queries about
assertions & evidence:
• Get all assertions that are supported by a
U.S. FDA regulatory guidance statement
• Are the evidence use assumptions are
concordant, unique, and non-ambiguous?
• Which assertions are supported/refuted by
just one type of evidence?
[Boyce2007, Boyce2009]
16. Micropublications Ontology (MP)
http://purl.org/mp
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications
17. Goal of this project
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
18. Modeling strategy
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence: MP
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
19. Modeling strategy
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence: MP
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web: OA
20. Quotes integrated (MP using OA)
http://purl.org/mp
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications
21. Enhancing the DIKB with MP and OA
1. Represent the overall argument of the paper
– Support & challenge relationships
– Data, methods, materials
2. Semantic tagging, so drugs & proteins can be
queried using knowledge from other sources
3. Make quotes actionable (highlight in orig doc)
4. Handle new competency questions
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33. Quote stored in OA, with link to source
ex:annotation-1
ex:body-1 ex:target-1
Predicate Object
rdf:type mp:Method
rdf:value (exact text)
about
Predicate Object
rdf:type oa:SpecificResource
oa:hasSource <http://dailymed…>
oa:hasSelector ex:selector-1
34. Quote stored in OA, with link to source
ex:annotation-1
ex:body-1 ex:target-1
Predicate Object
rdf:type mp:Method
rdf:value (exact text)
about
Predicate Object
rdf:type oa:SpecificResource
oa:hasSource <http://dailymed…>
oa:hasSelector ex:selector-1
ex:selector-1
Predicate Object
oa:prefix (preceding text)
oa:exact (exact text)
oa:postfix (following text)
35. New competency questions to answer
1. Finding assertions and evidence
• List all assertions that are not supported by evidence
– By data, by methods, by materials
• What is the in vitro evidence for assertion X? the in vivo
evidence?
– With provenance: Give me back the original data tables
2. Enabling updates
• List all evidence that has been flagged as rejected from
entry into the knowledge base
– By data, by methods, by materials
36. New competency questions to answer
3. Assessing the evidence
• Which research group conducted the study used for
evidence item X?
• What are the assumptions required for use of this
evidence item to support/refute assertion X?
– Without directly entering them
4. Statistics for analytics/KB maintenance
• Number of evidence items for and against each
assertion type
– By data, by methods, by materials
37. Modeling challenges
• To date, MP has not been used to represent
both unstructured text claims
("escitalopram does not inhibit CYP2D6")
and logical representation of text as
normalized subject-predicate-object
(nanopublication of statement)
• Efficient querying will be needed, even when
the evidence base scales. We are using an
iterative design-and-test approach.
38. Future work
• NLP support: Create a pipeline for extracting
potential drug-drug interaction (PDDI) mentions
from scientific & clinical literature
• Usability tests: Tools usable by domain experts
• NLP + "crowdsourcing" (distributed annotation)
• Resolving links to paywalled PDFs
39. Acknowledgements
• Funding
– ERCIM Alain Bensoussan fellowship Program
under FP7/2007-2013, grant agreement 246016
– National Library of Medicine (1R01LM011838-01)
• Thanks to the Evidence Panel of Addressing
PDDI Evidence Gaps: Carol Collins, Lisa Hines,
and John R Horn, Phil Empey
• Thanks to programmer Yifan Ning
Editor's Notes
20 min including questions
http://linkedscience.org/events/lisc2014/
Paper: http://jodischneider.com/pubs/lisc2014.pdf
Semantic web technologies can support the rapid and transparent validation of scientific claims by interconnecting the assumptions and evidence used to support or challenge assertions. One important application domain is medication safety, where more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions is needed. Exposure to potential drug-drug interactions (PDDIs), defined as two or more drugs for which an interaction is known to be possible, is a significant source of preventable drug-related harm. The combination of poor quality evidence on PDDIs, and a general lack of PDDI knowledge by prescribers, results in many thousands of preventable medication errors each year. While many sources of PDDI evidence exist to help improve prescriber knowledge, they are not concordant in their coverage, accuracy, and agreement. The goal of this project is to research and develop core components of a new model that supports more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions. Two Semantic Web models—the Micropublications Ontology and the Open Annotation Data Model—have great potential to provide linkages from PDDI assertions to their supporting evidence: statements in source documents that mention data, materials, and methods. In this paper, we describe the context and goals of our work, propose competency questions for a dynamic PDDI evidence base, outline our new knowledge representation model for PDDIs, and discuss the challenges and potential of our approach.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Statistics from Institute of Medicine Report Brief, July 2006 on Preventing Medication Errors
http://www.iom.edu/~/media/Files/Report%20Files/2006/Preventing-Medication-Errors-Quality-Chasm-Series/medicationerrorsnew.pdf
# How many pharmacokinetic studies in the DIKB could be used to support or refute
# an increases AUC assertion?
# How many AUC studies are in the DIKB that are based on data from the
# product label?
# Are the evidence use assumptions are concordant, unique, and non-ambiguous
# Get all assertions that are supported by an FDA guidance statement
# SHOW THE DISTRIBUTION OF THE LEVELS OF EVIDENCE FOR MECHANISTIC ASSERTIONS
# WHAT ASSERTIONS ARE SUPPORTED/REFUTED BY JUST ONE TYPE OF EVIDENCE?
## which items in the DIKB have evidence for and against that is both
## from product labeling?
# number of assertions in the system
# number of evidence items for and against
## find assertions that are not supported by evidence
# what single evidence items act as as support or rebuttal for multiple substrate_of assertions?
#remove a older version of a redundant evidence item
## change evidence for to evidence against
# has this evidence item been rejected
# what other assertions are being supported/challeged by this evidnece item?
Annotations in the data model are a set of RDF resources that connect some target to a set of resources that are in some way about it.
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. J. Biomedical Semantics 5: 28 (2014)
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. J. Biomedical Semantics 5: 28 (2014)
New competency questions handled:
Assertions that are not supported by evidence already in the evidence base, the evidence that should be checked most thoroughly (e.g. evidence that by itself supports multiple assertions), and the data, methods, and materials associated with a given evidence item as described in source documents.
1. Finding assertions:
(a) List all assertions that are not supported by evidence
(b) Which assertions are supported (or refuted) by just one type of evidence?
(c) Which assertions have evidence from source X (e.g., product labeling)
(d) Which assertions have both evidence for and evidence against from a
single source X?
2. Finding evidence:(a) List all evidence for or against assertion X (by evidence type, drug, drug
4.2
pair, transporter, metabolic enzyme, etc.)
(b) What is the in vitro evidence for assertion X? the in vivo evidence?
(c) List all evidence that has been flagged as rejected from entry into the the knowledge base
(d) Which single evidence items act as support or rebuttal for multiple as- sertions of type X (e.g., substrate of assertions)?
Assessing the evidence:
3. Understanding evidence coming from a given study:
(a) What data, methods, materials, are reported in evidence item X?
(b) Which evidence items are related to and follow-up on evidence item X?
(c) Which research group conducted the study used for evidence item X?
(d) AretheevidenceuseassumptionsforevidenceitemXconcordant?unique?
non-ambiguous?
4.3
Verifying plausibility of an evidence item:
(a) Has evidence item X been rejected for assertion Y? If so, why and by whom?
(b) Which other assertions are being supported/challenged by this evidence item?
(c) What are the assumptions required for use of this evidence item to sup- port/refute assertion X?
Checking assertions about pharmacokinetic parameters (i.e., area under the concentration time curve (AUC))
(a) How many pharmacokinetic studies used for evidence items in the DIKB
could be used to support or refute an assertion about pharmacokinetic
paramater X (e.g., ‘X increases AUC’)?
(b) How many pharmacokinetic studies in the DIKB used for evidence items
for assertion X are based on data from the product label?
(c) What is the result of averaging (or applying some other statistical oper- ation) to the values for pharmacokinetic parameter X across all relevant
studies used for evidence items?
Checking for differences in the product labeling:
(a) Are there differences in the evidence items that were identified across different versions of product labeling for the same drug?
(b) What version of product labeling was used for evidence item X? Original manufacturer or repackager? Most current label or outdated? Is the drug on market in country X or not? American or country X?
Supporting updates to evidence and assertions
1. Changing status of redundant and refuted evidence:
(a) Remove a older version of a redundant evidence item
(b) Change the modality of a supporting evidence item to be a refuting
evidence item
2. Updating when key sources change:(a) Get all assertions that are supported by evidence items identified from
an FDA guidance or other source document just released as an updated version.
4.4 Understanding the evidence base
1. Statistical information about the evidence base:
(a) Number of assertions in the system
(b) Number of evidence items for and against each assertion type
(c) Show the distribution of the levels of evidence for various assertion types
(e.g., pharmacokinetic assertions)
strength
Software responsive
For adding annotations: Existing MP plugin for Domeo
For viewing annotations: Want them highlighted in a web-based interface BUT Resolving annotations requires a method for pointing to paywalled/subscription PDF & HTML
An existing Micropublication plugin for Domeo [Ciccarese2014] is being mod- ified as part of the project. Our plan is to use the revised plugin to support the evidence board with the collection of the evidence and associated annotation data. It will also enable the broader community to access and view annotations of PDDIs highlighted in a web-based interface. We anticipate that this approach will enable a broader community of experts to review each PDDI recorded in the DIKB and examine the underlying research study to confirm its appropriateness and relevance to the evidence base.
The usability of the annotation plug-in is critically important so that the panel of domain experts will not face barriers to annotating and entering ev- idence. This will require usability studies of the new PDDI Micropublication plugin. Another issue is that many PDDI evidence items can be found only in PDF documents. Currently, the tool chain for PDF annotation is relatively weak: compared to text and HTML, PDF annotation tools are not as widely available and not as familiar to end-users. Suitable tools will have to be integrated into the revised plugin.
PDF documents may be in proprietary portals or academic library systems