LabeledIn : Cataloging Labeled Indications for Human Drugs

1. Stromectol is indicated for the treatment of Strongyloidiasis of the intestinal tract
and Onchocerciasis due to nematode parasite. This indication is based on
randomized double blind, placebo controlled and comparative studies.
Ketorolac tromethamine tablets are
indicated for short term management
of moderate to severe pain that
requires analgesia at the opioid level.
ACULAR ophthalmic solution is
indicated for the temporary relief of
ocular itching due to seasonal allergic
conjunctivitis. It is also indicated for the
treatment of postoperative
inflammation in patients who have
undergone cataract surgery.
LabeledIn: Cataloging Labeled Indications for Human Drugs
Ritu Khare1, Jiao Li1,2, Zhiyong Lu1
1National Center for Biotechnology Information, National Library of Medicine, NIH
2Institute of Medical Information, Chinese Academy of Medical Sciences
Background Semi-Automated Annotation Pipeline
Results: LabeledIn
Drug Label Selection
Manual Indication Annotation
(Experts A1 and A2 - Two Rounds)
Drug Indication
Resource
(LabeledIn)
#Drugs #Drug-
Indication
Pairs
IN (249) 1,318
IN+DF (611) 2,997
IN+DF+ST
(1,513)
7,805
Fig 3 The Annotation Framework with Three Components
LabeledIn Highlights
Computable Structured and
Normalized
Human-Validated Kappa agreement:
Round-1 77.48%
Round-2 88.35%
Source Linked Offsets in drug labels
Source Coverage ~50% Human Rx Drug
Labels
Drug Coverage >65% drug searches on
PubMed Health
Data Availability Multiple Formats
Related Publications
Khare R, Wei C-H, Lu Z, Annotating FDA Drug Labels for Drug Indications, AMIA Summits on Translational Sciences, 2014
Khare R, Li J, Lu Z, Toward Creating a Gold Standard of Drug Indications, IEEE International Conference on Health Informatics, 2013
Future Directions
Increase drug coverage: machine learning &
crowdsourcing
Integrate with other resources (e.g. EMRs)
Most existing “gold standard” repositories on drugs and their indications
are unstructured and do not support computer processing or semantic
interoperability. Text mining efforts to automatically extract structured
indications from free text are limited due to NLP challenges.
Fig 2 Using Existing Disease NER Tools for Indication Recognition
We propose a semi-automatic annotation framework and present a
unique resource – LabeledIn – which contains information about drugs
and their marketed indications and has the following key properties:
DailyMed (XML/HTML)
(FDA Drug Labels)
Motivation
Disease 1
Disease 2
Disease 3
Disease
Drug Indications
e.g. What are the indicated usages of
Fluoxetine
Disease Treatments
e.g. What drugs may be prescribed for
hypertension
Fig 1 Applications of Drug-Disease Treatment Relationships
Question and Answering
e.g. Google Knowledge Graph
Enrich Hyperlinks in
Online Resources
Computational Drug
RepurposingEMR Decision Support and
Error Control
1. Computable
 Structured
 Normalized (UMLS, RxNorm)
2. Accurate
 Source Linked
 Precise
 “Azithromycin 250 mg oral tablet” instead of
“Azithromycin”
 “Respiratory Tract Infections” instead of “Infections”
Our Contributions
Human
Judgment
Automatic Disease Recognition
IN = ingredient
DF = dose form
ST = strength
Clonazepam(Klonopin)
Azithromycin(Zithromax)
16,621
8,151
500
All Human Rx
250 most searched
ingredients
Representative labels
for annotation
DailyMed Current Medication Information
Round-1: Accept/Reject Pre-annotations
Round-2: Improve Previous Judgments
Fig 4 Applying the Annotation Framework on DailyMed August 2012 Version
Acknowledgments
This research was supported by the Intramural Research Program of the National Institutes of Health, National Library of Medicine.
The investigators would like to thank the three human annotators for their efforts.
Rejected Accepted
Additionally Suggested
Using Disease Recognition for Indication Extraction
Precision Recall F1-measure
0.55 0.94 0.69
ACULAR ophthalmic solution is
indicated for the temporary relief of
ocular itching due to seasonal allergic
conjunctivitis. It is also indicated for the
treatment of postoperative
inflammation in patients who have
undergone cataract surgery.
Ketorolac tromethamine tablets are
indicated for short term management
of moderate to severe pain that
requires analgesia at the opioid level.
Diltiazem hydrochloride extended release
capsules USP are indicated for the treatment of
hypertension. Diltiazem hydrochloride extended
release capsules is indicated for the management
of chronic stable angina and angina due to
coronary artery spasm.
Diltiazem hydrochloride extended release
capsules USP are indicated for the treatment of
hypertension. Diltiazem hydrochloride extended
release capsules is indicated for the
management of chronic stable angina and angina
due to coronary artery spasm.
Diltiazem hydrochloride extended release
capsules USP are indicated for the treatment of
hypertension. Diltiazem hydrochloride extended
release capsules is indicated for the management
of chronic stable angina and angina due to
coronary artery spasm.
Legend: Current Selected Not Selected
Legend: Previously Selected by A1 only
Previously Selected by A2 only
16181332 97
LabeledIn
Pre-annotations
Fig 5 Computer pre-annotations Vs. Human Annotations
Fig 6 Comparison of LabeledIn and SIDER 2
(50 drug labels)
LabeledIn
SIDER 2
92431384
56%
9%
14%
21%
Unique Pairs in
LabeledIn
23%
49%
12%
10% 3% 3%
Unique Pairs in SIDER
● More Specific
● CUI Different
● Disjoint Mention
● Missed by SIDER
● Other Context
● Generic Overlapping
● Less Specific
● Non-Disease
● CUI Different
● Side Effect
/Contraindication
x
x
#DrugLabel-Disease
Pairs
#DrugLabel-Disease Pairs
LabeledIn vs. Computer Pre-annotations LabeledIn vs. SIDER 2 Indications