I’ve been working with taxonomies for decades, with support from the company President , though it took awhile to call what we had a taxonomy. Still working on making data optimal for data scientists. Recently realized we have multiple taxonomies, with varying degrees of documentation.
Life Science specific applications include Drug discovery, Drug repurposing, Pharmacovigilance/adverse effects monitoring, Real world Evidence, Mapping complex disparate relationships (such as model organisms' data). I’m particularly interested in predictive analytics and trend analysis, with my projects still in very early stages.
I’m focusing on similarities between taxonomies and ontologies today, not the differences. I’m not going to talk about whether you should be using taxonomies or ontologies, or linked data or knowledge graphs. These are important discussions and other talked will consider them.. I’m also looking forward to hearing talks on machine learning, automatic tagging and governance as well.
Questions I most want to ask? Ones where the answer will surprise me.
The more we learn the more we realize we still have to understand. New insights don’t always invalidate old knowledge – but understanding becomes more and more granular. Taxonomies alone won’t solve all of these challenges, but they are an important part of the process.
I Some of my examples are life science specific, but a fair amount of my taxonomy work is more general as well.
A study of 500 plus biology papers published in a 20-year span suggested that up to 80% of raw data collected for studies in the early 1990s is lost, “mostly because no one knows where to find it.” Current Biology 2014Digital data are ephemeral ... “’homeless’ data quickly become no data at all.” Berman, Science 2019 Monya Baker, Nature 2016 survey of researchers. https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 Current Biology Jan 6 2014, described in Wiener-Bronner 2013 Atlantic article https://www.theatlantic.com/national/archive/2013/12/scientific-data-lost-forever/356422/ Who Will Pay for Public Access to Research Data? Francine Berman1, Vint Cerf2 Science 09 Aug 2013: Vol. 341, Issue 6146, pp. 616-617 DOI: 10.1126/science.1241625 https://science.sciencemag.org/content/341/6146/616.summary
Keep in mind the dangers of “overfitting” data.
“we are confident that the true cost is much higher than the estimated” PWC Study looked at time spent, cost of storage, license costs, research retraction, double funding, interdisciplinarity and potential economic growth.
Return on Investment of Natural Language Processing, Linguamatics paper cites time savings of 10x to 1000x or 1 FTE a year for every 10-12 drugs monitored, and costs savings of $40,000 by not investing in a project that would have had a negative outcome, or an improvement of $50-100K in risk adjusted revenue per disease area , productivity gains, discovery of 33 novel drug targets, applications for clinical trials and pharmacovigilance, automation of manual curation of publications and clinical trial endpoints, and re-use of existing data from clinical trials to speed up drug development. https://www.linguamatics.com/products/return-investment
And it’s even harder than I realized when I wrote the proposal for this conference. Learned at BioIT May 2019 just how difficult it is to engage the C-Suite.
Am using some of these sentences as scripts for in-house conversations now.
Some people want to rely only on algorithms and automation. I’m still advocating a hybrid approach, with varying success.
Consider collecting data on extent of existing problems with finding or reusing data.
People use an amazing variety of terms to describe terminology functions.
See the Go Fair website for more information on this set of principles with guidelines on how to make data FAIRer.
Many of the concepts I’m working with are cutting- to bleeding- edge and terminology is evolving organically. Lots of uncertainty Think of the image of shooting at a moving target – or as hockey great Wayne Gretzky said. “I just try to skate to where the puck is going to be.”
Do your homework. Know your audience. Think strategically. Are there detractors or skeptics? How can you address them?
Right now people seem willing to throw millions into machine learning or artificial intelligence – but are reluctant to invest in data readiness and data quality efforts. We’ve got to collaborate! We’ve got to share!
“Expect things to take even longer than you anticipate. We know less today than we will tomorrow (which means we know the least when we start” [Thanks to Terrell Russell of the iRODS ConsortiumUnderstand the workflows of the people whose problems you are trying to solve. Drastic workflow changes often means change will never happen.Users are much better at telling you what they don’t like than knowing what they really want, but may not be able to envision. Thank people for this feedback. Focus on 80/20 - actually 20/80. Definitely don’t aim for 100%.
Nature 2016 survey of researchers. https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 Current Biology Jan 6 2014, described in Atlantic article https://www.theatlantic.com/national/archive/2013/12/scientific-data-lost-forever/356422/ Who Will Pay for Public Access to Research Data? Francine Berman1, Vint Cerf2 Science 09 Aug 2013: Vol. 341, Issue 6146, pp. 616-617 DOI: 10.1126/science.1241625 https://science.sciencemag.org/content/341/6146/616.summary Life Science Leader 2019 March 1, “AI In Life Sciences: Seeing past the Hype” Francois Nicolas and comment by Christy Wilson Pistoia Alliance Ontologies Mapping https://www.pistoiaalliance.org/projects/current-projects/ontologies-mapping/
Ontology mapping for semantically enabled applicationsSummary of recent progress from thesauri, taxonomies to ontologies “when biomedical research is under a deluge of an increasing amount and variety of data… Semantic alignment and data standardization are vital to solve if we are going to harness modern technologies such as machine learning" Drug Discovery Today May 2019
How to create a taxonomy for management buy-in
You thought taxonomies were hard?
How to create a taxonomy
for “management buy-in"
Mary Chitty MSLS
London, Oct 15, 2019
Needham MA USA
Where I’m coming from
2019 Taxonomy now 1,600+ terms and
growing. Company is now mid-size.
2018 Signed contract with OntoForce
to use Disqover search software.
Acquired Artificial Intelligence
and Internet of Things
companies. Hired several data
• Data visualization
• Search, Semantic search and Search Engine
• Statistical analysis
• Text mining
• “Ontologies offer advantages over other knowledge
systems—they enable both computational use and
human understanding, they can …include rich
vocabularies of labels, synonyms, and textual
definitions. If these are desirable selection criteria, then
an ontology should be considered. ”
• “Ontologies do also come with computational
overheads, however, and can be complex to understand.
Other resources such as a vocabulary do not offer the
sorts of classification and rich computational
descriptions of an ontology but are often much simpler
to understand. Let your requirements guide you;
ontologies are not a panacea—sometimes
one isn’t needed at all. ”
• Malone 2016
Value adds of taxonomies/ontologies:
Data harmonization especially of named entities
Identification of data errors or inconsistencies
Search/data navigation improvements
Collaborative filtering/recommendation engines
Validation of correlations to examine possible causalities
Enabling previously unaskable questions/use cases
Ambiguous and inconsistent data
Missing or unfindable data
Scaling up data processes
Sustainable maintenance – perhaps the biggest challenge of all!
Primary challenges are as much cultural as technological
Life sciences challenges include:
Relatively sparse data compared to other domains such as financial
Highly dimensional data with many variables (complex to chaotic)
Inherently noisy biological data. (Increasingly studied at the single cell or
gene expression level).
Data on longitudinal health outcomes limited by HIPAA & other privacy
regulations, but crucial for evidence based medicine validation.
In our era of big data, the irony is we don’t have enough readily usable life
• More than 70% of researchers have tried but
failed to reproduce experiments. More than
half failed to reproduce their own
experiments. Baker, Nature 2016
• “replication alone will get us only so far (and)
might actually make matters worse… an
essential protection against flawed ideas … is
the strategic use of multiple approaches to
address one question. Each approach has its
own unrelated assumptions, strengths and
weaknesses. Results that agree across different
methodologies are less likely to be artefacts”.
Wikipedia “replication crisis”
Educating decision makers is an ongoing process, even
with CXOs who value taxonomies/ontologies.
But stakeholders are often skeptical about investing in
taxonomies or ontologies.
Minimum of 10.2 billion Euros per year.
Cost of not having FAIR research data:
"The first layer of the semantic Web consists of ontologies and
taxonomies ... A huge amount of this is being done very desperately
in the realm of biotech, for the human genome and new drug
Tim Berners Lee, August 30, 2001 keynote at Software
Development East in Boston.
“Semantics is fundamentally not an information technologies issue
…it originates out of the need for groups of individuals to work
together towards common goals … must agree upon a set of
meanings around terminologies, concepts, relations and actions …
a lot of confusion arises before people realise whether they are
talking about the same or different things."
Eric Neumann, Applying the semantic web to drug discovery and
development. Drug Discovery World Fall 2005
is not a new
“a month in the lab can often save an hour in the
library” Attributed to chemist Frank Westheimer
“Institutions either underestimate the resources
needed to do this work , or they are daunted by
the entire prospect ... Honestly, very little data
will ever be reused. ” personal communication,
Juliane Schneider, eagle-i, Harvard Catalyst
Best Practical Advice I’ve come across
“One of my mantras is always start small. Show some win in some
small domain. Don’t under any circumstance start with saying I’ll just build you this
enormous ontology for the next two years …then your world will be better”. .. Just
say I’m going to build this tiny little ontology and enable this small application over here
… Always making sure my small ontology is enabling the small win.”
“Question: how do you encourage semantic modeling? Answer: First I compliment them,
and say what you’ve done is a great starting point – because they have actually started …
I try to find a couple of structured retrieval applications that they really want to do but
their current markup is not allowing ….find two compelling examples … make sure
that we’ve got a deliverable in a month or a short period of time where they can do the
one trial thing that adds value. Kind of get them on the slippery slope so that they’re’ the
owner and they want to do it themselves.” Deborah McGuinness keynote speech 2004
Taxonomy Use Cases
Amazon I spoke at Taxonomy Boot Camp
2017 in Washington DC and learned that
Amazon has taxonomists on 24 hour
emergency call, for when people can’t find
their products online.
Netflix “[Netflix]paid people to watch films
and tag them with all kinds of metadata.
This process is so sophisticated and precise
that taggers receive a 36-page training
document that teaches them how to rate
“[Netflix] even offered a $1 million prize to the team that could design
an algorithm that would improve the company's ability to predict how
many stars users would give movies. It took years to improve the algorithm
by a mere 10 percent. The prize was awarded in 2009, but Netflix never
actually incorporated the new models. ” Madrigal, Atlantic 2014
Best business case for
My Taxonomy Case Studies in-house
Map industry verticals to
company specific verticals
Question: Can we use any
existing taxonomies such as
NAICs codes and CrunchBase
to automate? How to
integrate existing in-house
database with newly
databases? Work in
Job title functions and
seniorities for people in
Data scientist automated. At
least 80% assigned now.
Customizable for use by
Phase 1 just completed. Still
reviewing and fine-tuning.
Job departments for people
Similar to job titles. Phase 1
Ontoforce internal data
Project to enable improved
access to existing in-house
inconsistencies with labels
and tables. Identified data
quality issues to address.
Working on training users,
Need to add more
changeability to existing
structure. Starting to see how
trend analysis may be
possible. Work in progress.
for starting a
• Consider a pilot/proof of concept.
• Start small because that will be easier to evaluate
Don’t try to “boil the ocean”.
• Choose a variety of data complexity. Think about
degrees of granularity when drafting categories.
• Which categories might you want to aggregate?
• Which related concepts might you want to segment
further? [Phase 2]
• Are there assumptions or implicit biases you might
be making without realizing?
• Solicit feedback from diverse stakeholders as an
Don’t be surprised while building your pilot project :
Terminology consensus is challenging at best.
naming, tagging, tables,
Many ways to express
“Biologists would rather
share a toothbrush than
gene names” Michael
Ashburner GeneSeer: A
sage for gene names and
"Biologists would rather
share a tooth brush than
data” Carol Goble
Ashburner” Keynote EGEE
Trying for consensus often
gets very emotional,
challenging – and
•First step in
(re)using data is to
Metadata and data
should be easy to
both humans and
computers. … an
•Once the user
finds the required
needs to know
how can they be
•Data usually need
to be integrated
with other data …
•Ultimate goal of
FAIR is to optimise
the reuse of
and data should
so that they can be
Keep in mind for management presentation
FAIR data can help
European Commission and US National Institute of Health have allocated considerable resources to making data FAIRer.
t may ask
“Why can’t I
• Search works best if you know what you’re looking
for exists AND what to call it.
• Taxonomies are more useful if you’re not sure
what you want exists, OR you don’t know what to
call it, AND/OR there are multiple ways to express
• Harness the power of serendipity with
taxonomies. They give people a sense of whether
the “scent of information” is promising.
Look for and document
Employee time savings
Added competitive advantages
Sponsors, champions and/or influencers
Clear executive summary, with KPIs Key Performance
Indicators, milestones, values, costs
1-2 pages Utilize url links if needed
Align early. Align often.
Ask for feedback – it’s a way of getting buy-in.
Leave room for suggestions.
Be aware of other company initiatives.
But what else needs to
“[T]here is a lot of work that needs doing to prepare the data sets for these technologies
… a disproportionate amount being invested in the technologies as opposed to investing
in "data-readiness… It's just not a slam dunk to mash up a lot of data and think it
will work. … The AI solution may help accelerate some tasks, but human expertise may
be required for the broad scope of what is needed. “ Nicholas 2019
• Open Science
"Is any lifetime long enough these days to learn everything needed to get a drug to
market and keep it there? "
• More need than ever to collaborate to share knowledge, especially pre-competitively.
1. Aim first for quick wins with low hanging fruit.
2. Bundle stakeholders valued wants with items you can expect they will
3. Seek out allies to get shared buy-in for sustainable justification.
4. Pareto Principle 80% of effects come from 20% of effort.
5. Expectations/change management are crucial skills to cultivate.
6. Collect metrics (quantitative/qualitative) to measure progress, so you know
when you’ve made some.
7. Recognize some challenges haven’t been resolved by anyone yet.
Resources to use todayChitty, Mary, Ontologies & Taxonomies glossary & taxonomy, 2019 with 40 plus ontology definitions, 15 taxonomy definitions http://www.genomicglossaries.com/content/ontologies.asp
Heath, Chip and Dan, Switch: How to Change Things When Change is Hard, 2010 https://heathbrothers.com/books/switch/
McGuinness, Deborah, Ontology Development 101, A Guide to creating your first ontology http://www.ksl.stanford.edu/people/dlm/papers/ontology-tutorial-noy-mcguinness-abstract.html
Research Data Alliance https://www.rd-alliance.org/ a research community organization started in 2013 by the European Commission, US National Science Foundation, US National Institute of Standards and
Technology, Australian Department of Innovation.
How Netflix Reverse-Engineered Hollywood, Atlantic, 2014 https://www.theatlantic.com/technology/archive/2014/01/how-netflix-reverse-engineered-hollywood/282679/
Life Science specific
BioPortal https://bioportal.bioontology.org/ repository of biomedical ontologies has almost 800 ontologies, mapping from ontologies to I2B2 http://i2b2.bioontology.org/
Malone, James et. al. Ten Simple Rules for selecting a Bio-Ontology, PLoS Comput Biol 12(2), 2016: e1004743. https://doi.org/10.1371/journal.pcbi.1004743
National Center for Biomedical Ontologies NCBO BioPortal Ontology to i2b2 File Mappings http://i2b2.bioontology.org/
Pistoia Alliance, Ontologies Guidelines for Best Practices to support practical application and mapping, 2016 https://pistoiaalliance.atlassian.net/wiki/spaces/PUB/pages/43089942/Ontologies+Guidelines+for+Best+Practice
Berneres Lee 2001 http://www.sdgnews.com/sd2001es_006/sd2001es_006.htm no longer on web
Neumann, 2005 https://www.ddw-online.com/informatics/p148329-applying-the-semantic-web-to-drug-discovery-and-development.html
Michael Ashburner GeneSeer: A sage for gene names and genomic resourcesBMC Genomics. 2005; 6: 134. 2005 Sep 21. doi: 10.1186/1471-2164-6-134
Carol Goble “purposely misquoting Michael Ashburner” Keynote EGEE 2006
Interoperability With Moby 1.0 - It's Better Than Sharing Your ...
Semantic alignment and data standardization are vital to solve if we are going to harness modern technologies such as machine learning”
Ian Harrow1 Rama Balakrishnan2 Ernesto Jimenez-Ruiz34 Simon Jupp5 Jane Lomax6 Jane Reed7 Martin Romacker8 Christian Senger9 Andrea Splendiani10 Jabe Wilson11 Peter Woollard12 Drug Discovery Today May 2019 https://www.sciencedirect.com/science/article/pii/S1359644618304215
Nicolas 2019 Life Science Leader 2019 March 1, “AI In Life Sciences: Seeing past the Hype” Francois Nicolas and comment by Christy Wilson https://www.lifescienceleader.com/doc/ai-in-life-sciences-seeing-past-the-hype-0001
Many people have participated in this ongoing project. I’m grateful for their work, insights and
& Cambridge Healthtech
•Phillips Kuhl, President
Services Analyst & IT
Sanaye Bartlett, Data
Analyst & Project Manager
Director of Product
CII Disqover Team
•Kaitlyn Barago, Associate
•Nancy Clarke, Data
Director New Initiatives
•Jaime Parlee, Director
•Craig Wohlers, Manager
•Hans Constandt, CEO
•Filip Pattyn, Scientific Lead
Customer Data Scientist
•Berenice Wulbrecht, Data
Science Director, Systems
•Ingrid Akerblom, IFA
John Aubrey, Vertex
Mark Burfoot, Novartis
Jane Lomax, SciBite
•Eric Neumann, Akidata LLC
•Juliane Schneider, eagle-i,
•Ted Slater, PaaS, Elsevier