My ontology is better than yours! Building and evaluating ontologies for integrative research
Upcoming SlideShare
Loading in...5
×
 

My ontology is better than yours! Building and evaluating ontologies for integrative research

on

  • 945 views

 

Statistics

Views

Total Views
945
Views on SlideShare
934
Embed Views
11

Actions

Likes
1
Downloads
14
Comments
0

6 Embeds 11

https://si0.twimg.com 3
http://www.linkedin.com 3
https://www.linkedin.com 2
http://tweetedtimes.com 1
http://localhost 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

My ontology is better than yours! Building and evaluating ontologies for integrative research My ontology is better than yours! Building and evaluating ontologies for integrative research Presentation Transcript

  • Introduction Biomedical ontology Use case: pharmacogenomics Outlook My ontology is better than yours! Building and evaluating ontologies for integrative research Robert Hoehndorf Department of Genetics University of Cambridge Bio-Ontology SIG
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookTranslational research National Cancer Institute: Translational research transforms scientific discoveries arising from laboratory, clinical, or population studies into clinical applications to reduce [disease] incidence, morbidity, and mortality.
  • slide by Robert Stevens View slide
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies Gruber (1993): An ontology is the explicit specification of a conceptualization of a domain. controlled vocabularies hierarchically organized facilitate data integration View slide
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies Individual Physical object Quality Function Process ChEBI Ontology Molecule Gene Sequence Ontology Transcript GO-CC Organelle Celltype Gene Ontology Cell Phenotype Tissue Ontology Organ Anatomy Ontology Body Population
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies How can we find the “best” ontology? How can we develop the “best” ontology?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesOntology evaluation
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesEvaluation criteria ontology design principles rooted in best practices philosophy logic ontology engineering linguistics community agreement community requests peer review
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookOntologyOntology evaluation definitions singular nouns common relations single is-a hierarchy orthogonality realism ...
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies Most ontology evaluation criteria are intrinsic criteria and evaluate what ontologies are.
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologies Most ontology evaluation criteria are intrinsic criteria and evaluate what ontologies are. How can we evaluate what ontologies do?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesA functional perspective
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesEvaluation criteria criteria from software engineering, etc. user study unit tests complexity ...
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesA functional perspective
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookBiomedical ontologiesEvaluation criteria criteria from biology experiments statistics (p-values) comparison to gold/silver standard ...
  • PharmacogenomicsPharmacogenomics databases
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookResearch questions drug discovery drug repurposing drug response drug pathways disease pathways causal mutations
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookResearch questions drug discovery drug repurposing drug response drug pathways disease pathways causal mutations
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookTraditional approaches to drug repurposing drug target identification models of drug binding experiment design and execution (e.g., binding assays) analysis and interpretation of experiment results
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookIntegrative approaches to drug repurposing SIDER text mining of drug labels side-effect similarity UMLS PREDICT disease–disease similarity drug–drug similarity disease phenotypes, gene functions, side effects, chemical structure, protein interactions, text mining HPO, MESH, GO OFFSIDES adverse event reports ATC, UMLS
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookPharmacogenomics Can we get some novel information about drug indications (and causal mutations) by analyzing experimental data from animal models?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookApproach
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookApproach
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookRelevant ontologies Mammalian Phenotype Ontology 9,161 classes manually developed annotation of animal models formal (EQ) definitions Human Phenotype Ontology 9,796 classes manually developed annotation of diseases formal (EQ) definitions
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookChallenges 1 comparison of human and mouse phenotypes cross-species integration how do we represent phenotypes? 2 computation of similarity semantic similarity based on ontology taxonomy which ontology do we use for computing similarity?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookCross-species phenotype integration representation of MP and HPO phenotypes PATO-based formal definitions GO homologous and analogous anatomical structures (UBERON) aim: cross-species integration of phenotypes
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookWhat are phenotypes and how do we represent them (forcross-species integration)? Abnormal appendix: E=Appendix, Q=Abnormal representation: appendix with quality Abnormal quality Abnormal of some appendix organism with appendix that has quality Abnormal ... inheritance of phenotypes across parthood Abnormality of tip of appendix subclass of Abnormality of appendix? absence of appendix
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity Semantic similarity results depend on the number of distinctions made by ontology developers the kind of distinctions made by ontology developers the data that is analyzed the similarity measure
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity Should we compute phenotypic similarity based on the Human or the Mammalian Phenotype Ontology (or both)? How can we compare the results?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookOntology design decisions can be resolved empirically! no a priori “right” way to represent phenotypes focus on scientific results, not representation evaluation: empirical objective quantitative external
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookOntology design decisions can be resolved empirically! finish the analysis use known gene–disease associations as gold standard use FDA-approved drug indications as gold standard compare analysis results against gold standard
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity over phenotype ontologies measuresphenotypic similarity semantic similarity pairwise comparison of disease and animal phenotypes IC (x) x∈Cl(P)∩Cl(D) sim(P, D) = IC (y ) y ∈Cl(P)∪Cl(D)
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookPhenomeNET compares phenotypes across species ranking of gene for each disease candidate genes for disease
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookStatistical testing to rank drug–disease pairs one-sided Wilcoxon signed rank test result: ranking of drugs for each disease based on p-value low p-value: mutations in mouse genes associated with a drug result in phenotypes that are very similar to a disease phenotype high p-value: genes uniformly distributed across ranks
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookReceiver Operating Characteristic Source: Wikipedia
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookGene-disease associations PhenomeNet initial 1 0.9 0.8 0.7 True Positive Rate 0.6 AUC: original 0.68 0.5 0.4 0.3 0.2 0.1 x original 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 False Positive Rate
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookGene-disease associations PhenomeNet improved 1 0.9 0.8 0.7 AUC (original): 0.68 True Positive Rate 0.6 AUC (latest): 0.89 0.5 0.4 0.3 0.2 0.1 x original latest 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 False Positive Rate
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookGene-drug associations PhenomeDrug initial 1 0.9 0.8 0.7 True Positive Rate 0.6 AUC: original 0.61 0.5 0.4 0.3 0.2 0.1 x original 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 False Positive Rate
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookGene-drug associations PhenomeDrug improved 1 0.9 0.8 0.7 AUC (original): 0.61 True Positive Rate 0.6 AUC (latest): 0.67 0.5 0.4 0.3 0.2 0.1 x original latest 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 False Positive Rate
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookRepresentation of phenotypes for cross-species integration ’Abnormality of appendix’ EquivalentTo: has-part some (part-of some (Appendix and has-quality some Quality)) organism-centric approach (has-part some) transitivity over parthood (part-of some) Quality used as indicator of abnormality use of OWL EL
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookRepresentation of phenotypes for cross-species integration ’Large appendix’ EquivalentTo: has-part some (Appendix and has-quality some ’Increased size’) organism-centric approach (has-part some) no transitivity over parthood use of OWL EL
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookAbsence ’Absence of appendix’ EquivalentTo: has-part some (Appendix and has-quality some Absent) subclass of Abnormality of appendix use of OWL EL
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity Should we compute phenotypic similarity based on the Human or the Mammalian Phenotype Ontology (or both)? How can we compare the results?
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity Computation of semantic similarity using the Mammalian Phenotype Ontology improves the analysis results. problem specific depending on mouse data depending on the approach depending on similarity measure depending on gold standard dataset
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookConclusion Quantitative, external evaluation can improve ontologies and ontology-based analysis methods.
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookAnnotation Definitions: intrinsic: having definitions Aristotelian definitions external: having definitions that are easily understandable having definitions that improve annotation consistency criteria: measure annotation consistency user study Dolan, M. E., et al. A procedure for assessing GO annotation consistency. Bioinformatics 21, i136–i143 (2005).
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookAnnotation Labels: intrinsic: singular nouns reference to universals external: use of common, widely used terms use of unambiguous terms criteria: measure annotation consistency user study recall in text Yao, L., et al. Benchmarking Ontologies: Bigger or Better? PLoS Comput Biol 7, e1001055 (Jan. 2011).
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookKnowledge bases and querying Queries: intrinsic: use of OWL use of specific relations use of upper level ontology consistency external: retrieve correct answers retrieve relevant answers criteria: user study (to evaluate query answers) test set comparison to gold standard Boeker, M., et al. Unintended consequences of existential quantifications in biomedical ontologies. BMC Bioinformatics 12, 456 (2011).
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookConclusions My ontology is better than yours.
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookConclusions My ontology is better than yours. My ontology can do some things better than your ontology.
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookConclusionsQuantitative criteria Empirical, objective, quantitative, application-based evaluation will allow us to systematically improve ontologies for science.
  • Thank you for your attention
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity
  • Introduction Biomedical ontology Use case: pharmacogenomics Outlook 1Semantic similarity: 12
  • Introduction Biomedical ontology Use case: pharmacogenomics OutlookSemantic similarity
  • Introduction Biomedical ontology Use case: pharmacogenomics Outlook 4Semantic similarity: 12