Abstract: http://j.mp/1MhWWei Healthcare applications now have the ability to exploit big data in all its complexity. A crucial challenge is to achieve interoperability or integration so that a variety of content from diverse physical (IoT)- cyber (web-based)- and social sources, with diverse formats and modality (text, image, video), can be used in analysis, insight, and decision-making. At Kno.e.sis, an Ohio Center of Excellence in BioHealth Innovation, we have a variety of large, collaborative healthcare/clinical/biomedical projects, all involving domain experts and end-users, and access to real world data that include: clinical/EMR data (of individual patients and that related to public health), data from a variety of sensors (IoT) on and around patients measuring real-time physiological and environmental observations), social data (Twitter, Web forums, PatientsLikeMe), Web search logs, etc. Key projects include: Prescription drug abuse online-surveillance and epidemiology (PREDOSE), Social media analysis to monitor cannabis and synthetic cannabinoid use (eDrugTrends), Modeling Social Behavior for Healthcare Utilization in Depression, Medical Information Decision Assistant and Support (MIDAS) with application to musculoskeletal issues, kHealth: A Semantic Approach to Proactive, Personalized Asthma Management Using Multimodal Sensing (also for Dementia), and Cardiology Semantic Analysis System (with applications to Computer Assisted Coding and Computerized Document Improvement). This talk will review how ontologies or knowledge graphs play a central role in supporting semantic filtering, interoperability and integration (including the issues such as disambiguation), reasoning and decision-making in all our health-centric research and applications. Additional relevant information is at the speaker’s HCLS page. http://knoesis.org/amit/hcls

1. 1
Ontology-enabled Healthcare Applications
Exploiting Physical-Cyber-Social Big Data
Ontology Summit for the Health Care Track on Semantic Integration, 7 April 2016
Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing:
An Ohio COE on BioHealth Innovation
Wright State University
Special thanks: Sujan Parera

2. 33%
35%
32%
Kno.e.sis: Ohio Center of Excellence
in Knowledge-enabled Computing
DoD & Industry
• Metabolomics & Proteomics
• Medical Info Decisions
• Human Detection
on Synthetic FMV
• Sensor & Information
• Material Genomics
• Cardiology Semantic Analysis
NIH: National Inst. of Health
• kHealth - Asthma
• eDrug Trends
• Depression on Social Media
• Drug Abuse Early Warning
NSF: National Science Foundation
• Harassment on Social Media
• Citizen & Physical Sensing
• Twitris - Collective Intelligence
• Aerial Surveillance
• Visual Experience
• Web Robot Traffic
Kno.e.sis’ research in World Wide Web ranks Wright State University among the top 10 organizations in the world based on 10-yr
impact. Its total budget for currently active projects is $11,443,751, with $5,912,162 for new projects starting after July 2015. The
significant majority of funds are highly competitive federal grants. World-class research is complemented by exceptional student
outcomes and commercialization with local economic impact.
As an Ohio COE on Bio Health
Innovation, Kno.e.sis conducts
research leading to building intelligent
systems for clinical, biomedical, policy,
and epidemiological applications.
Example clinical/healthcare
applications include major diseases
such as asthma, depression,
cardiology, dementia and GI.
This is complemented by social and
development challenges such as
marijuana legalization policy,
harassment on social media, gender-
based violence, and disaster
coordination.
60+ Funded Students
• 40 PhD
• 16 MS
• 5 BS

3. 3
Collaborators

4. Projects @ Kno.e.sis
Image Credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png , http://rotwnews.com/wp-
content/uploads/2014/04/DRUG-TRENDS-Talk.jpg, http://slapheadmarketing.com/wp-content/uploads/2012/05/Forum-Marketing.jpg
eDrugTrends is social media data analytics platform to monitor the cannabis and
synthetic cannabinoids usage. It uses social media and Web forums data to: 1)
Identify and compare trends in knowledge, attitudes, and behaviors related to
cannabis and synthetic cannabinoid, and 2) Identify key influencers in cannabis and
synthetic cannabinoid-related discussions on Twitter.
eDrugTrends
Data Sources
Project Wiki
Daily average content:
Tweets: 135,553
Forum Posts*: 8,899
Total: 144,452
* Bluelight, Drugs-forum, and Reddit

5. Projects @ Kno.e.sis
Image Credits: https://i.ytimg.com/vi/GOK1tKFFIQI/maxresdefault.jpg, http://slapheadmarketing.com/wp-content/uploads/2012/05/Forum-
Marketing.jpg, https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://encrypted-
tbn1.gstatic.com/images?q=tbn:ANd9GcSucmCuyWvX4dFHv5XvS3KzjvD11hC8HwK9N4004LnBZOGLOgf6,
http://www.crmchealth.org/sites/default/files/images/medical-records/Medical_Records_0.png?1314713869
Identifying combinations of online socio-behavioral factors and
neighborhood environmental conditions that can enable detection of
depressive behavior in communities and studying access and
utilization of healthcare services
Depression Behavior
Data Sources
Electronic Medical Records
Public Surveys
Project Wiki
Depending on collection method,
We get 7-17K tweets per day, and
Have 800K to 18M total tweets
in several months.

6. Projects @ Kno.e.sis
This project seeks to understand and satisfy users’ need for keeping track of new
information in healthcare and well-being. The project harvest collective
intelligence to identify high quality, reliable and informative healthcare content
shared over social media based on following analysis: Text Analysis, Semantic
analysis, Reliability analysis, Popularity Analysis.
Social Health Signals
Data Sources
Image Credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://encrypted-
tbn2.gstatic.com/images?q=tbn:ANd9GcS6qI3Z_Y0Uh0sPNCgy0J_0d66-5NsCwK3VqWsIkAKRmqjTSXK0uA
Project Wiki

7. kHeath analyzes both active and passive observations of the patients to generate the
alarms that helps to improve health, fitness, and wellbeing of the patient. It uses
Semantic Sensor Web technology, Semantic Perception, and Intelligence at the Edge to
enable sophisticated analysis of personal health observations.
kHealth
Projects @ Kno.e.sis
Data Sources
image credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, https://www.cooking-
hacks.com/media/cooking/images/documentation/e_health_v2/e_health_sensors_small.png,
http://www.co.freeborn.mn.us/ImageRepository/Document?documentID=483
Public Health APIs
Project Wiki

8. Projects @ Kno.e.sis
Monitor the health status of the military personnel in training period through self-
reported fitness notes and continuous monitoring with body sensors. The
collected data is used to assess the health status of the person and suggest
exercise regimen change or treatment plans if needed.
MIDAS
Data Sources
image credits: https://www.cooking-hacks.com/media/cooking/images/documentation/e_health_v2/e_health_sensors_small.png,
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcROFoUIaLPDWcvcCmoi1-sl8Bl3CPUtZooX5HHPuDiQKGI7oFZfuQ
Self-reported Data
Project Wiki

9. Projects @ Kno.e.sis
PREDOSE developed techniques to facilitate prescription drug abuse epidemiology,
related to the illicit use of pharmaceutical opioids. PREDOSE is designed to capture
the knowledge, attitudes and behaviors of prescription drug abusers through the
automatic extraction of semantic information from social media.
PREDOSE
Data Sources
image credits: https://www.boxchilli.com/wp-content/uploads/2014/07/socialmediamarketing.png, http://slapheadmarketing.com/wp-
content/uploads/2012/05/Forum-Marketing.jpg, https://encrypted-
tbn3.gstatic.com/images?q=tbn:ANd9GcTrMsVTVc6RJrWZtst5ZTILWoD83HO0DPbj3I89YSqMiNRdwI7S
Project Wiki

10. Projects @ Kno.e.sis
The scientific analysis of the parasite Trypanosoma cruzi (T. cruzi), the principal
causative agent of human Chagas disease, is the driving biological application of
this project. We developed and deployed a novel ontology-driven semantic
problem-solving environment (SPSE) for T.cruzi
SPSE – T.cruzi
Data Sources
image credits: https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcRB-YQT3LWMXm9vfv3IdclcMDjP-_ChizcFMw53OAkptnHdaUAn6w,
http://www.clfs.umd.edu/biology/machadolab/images/trypanosoma.jpg, http://web.eecs.umich.edu/~dkoutra/courses/W16_484/
Public & Private Databases
(Uniprot, GO, KEGG,
TriTrypDB
Project Wiki
experimental data from mass spectrometry
and microarray experiments
Textual Data

11. Ontologies Developed at Kno.e.sis
• Drug Abuse Ontology – 83 classes, 37 properties
• Depression Insight Ontology – ongoing work
• Healthcare Ontology/ezDI Knowledge Graph – proprietary
• Human Performance and Cognition “Ontology” – 2 million entities, 3
million facts (HPCO)
• Ontology for Parasite Lifecycle – 360 classes, 12 properties (BioPortal)
• Parasite Experiment Ontology – 142 classes, 40 properties (BioPortal)
• Provenir Ontology - 88 classes, 23 properties (Provenir) – a key input to
W3C provenance work
Earlier at UGA: ProPreO (500+ classes), GlycO,…

12. Semantic Filtering
Data Integration
Knowledge Enrichment
Entity Annotation
Triple Extraction
Sentiment Analysis/
Intent Mining/
User Modeling
r1
Search/Browsing/S
ummarization/
Trend/Analysis/Pre
diction
Knowledge Base Usages @ Kno.e.sis
Data alone is not enough.
KB+NLP+ML

13. Explanation
Module
Explained?
Yes
No
Hypothesis
Filtering
Hypothesis
Generation
Hypothesis
with High
Confidence
D
D D
DD
D
Patient Notes
UMLS
Knowledgebase Enrichment
Sujan Perera, Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth, and Sahas Nair. "Semantics driven approach for knowledge acquisition from
EMRs." Biomedical and Health Informatics, IEEE Journal of 18, no. 2 (2014): 515-524.

14. Knowledgebase Enrichment
● Knowledge in a given knowledge base may not always sufficient
● Acquiring required knowledge in some domains is a tedious task
● Data available for a particular domain may contain required knowledge
● Partial knowledge about the domain can be used to efficiently acquire
domain knowledge from data that can fill existing gaps in a knowledge
base
Data

15. Data Integration with Ontologies
UniPort
Internal
Lab Data
T.cruzi DB
NCBI Data Sources
PubMed
T.cruzi immunology ontology Parasite Experiment ontology T.cruzi life cycle ontology
Aligned Ontologies

16. ● Qualitative studies such as telephonic survey which suffer from limited population
coverage and large temporal gaps.
● To address limitations of the qualitative studies, researchers have used various
data sources such as social media (e.g. Twitter), web search logs, and
neighborhood factors.....but in silos
Depression
Social Media
Web Search
log
Neighborhood
factors
EHR data
Depression Behavior

17. Depression Behavior

18. PREDOSE

19. Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Background
Knowledge
tweet reporting pollution level
and asthma attacks
Acceleration readings from
on-phone sensors
Sensor and personal
observations Signals from personal,
personal spaces, and
community spaces
Risk Category assigned by
doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Well Controlled - continue
Not Well Controlled – contact nurse
Poor Controlled – contact doctor
kHealth

20. Dealing with Heterogeneity
He showed shortness of breath in last visit
Dyspnea was observed in his last visit
It is observed that patient has labored breathing
The patient was breathing comfortably in room air
He showed short of breath in last visit
C0013404
shortness of breath dyspnea
Labored or difficult breathing associated with a
variety of disorders, indicating inadequate
ventilation or low blood oxygen.
rdfs:labelrdfs:label
is_defined_as
Expressing the Shortness of Breath
explicit mention
syntactic variation
synonym
positive implicit mention
negative implicit mention
individual literal

21. I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do
shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and
tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could
feel the bupe working but overall the experience sucked.
Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4
mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty
damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.
Codes Triples (subject-predicate-object)
Suboxone used by injection, negative experience Suboxone injection-causes-Cephalalgia
Suboxone used by injection, amount Suboxone injection-dosage amount-2mg
Suboxone used by injection, positive experience Suboxone injection-has_side_effect-Euphoria
experience sucked
feel pretty damn good
didn’t do shit
feel great
Sentiment Extraction
bad headache
+ve
-ve
Triples
DOSAGE PRONOUN
INTERVAL Route of Admin.
RELATIONSHIPS SENTIMENTS
DIVERSE DATA TYPES
ENTITIES
I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do
shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and
tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could
feel the bupe working but overall the experience sucked.
Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4
mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty
damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.
I was sent home with 5 x 2 mg Suboxones. I also got a bunch of phenobarbital (I took all 180 mg and it didn't do
shit except make me a walking zombie for 2 days). I waited 24 hours after my last 2 mg dose of Suboxone and
tried injecting 4 mg of the bupe. It gave me a bad headache, for hours, and I almost vomited. I could
feel the bupe working but overall the experience sucked.
Of course, junkie that I am, I decided to repeat the experiment. Today, after waiting 48 hours after my last bunk 4
mg injection, I injected 2 mg. There wasn't really any rush to speak of, but after 5 minutes I started to feel pretty
damn good. So I injected another 1 mg. That was about half an hour ago. I feel great now.
Buprenorphine
subClassOf
bupe
Entity Identification
has_slang_term
SuboxoneSubutex
subClassOf
bupey
has_slang_term
Drug Abuse Ontology (DAO)
83 Classes
37 Properties
33:1 Buprenorphine
24:1 Loperamide

22. Ontology Lexicon Lexico-ontology Rule-based Grammar
ENTITIES
TRIPLES
EMOTION
INTENSITY
PRONOUN
SENTIMENT
DRUG-FORM
ROUTE OF ADM
SIDEEFFECT
DOSAGE
FREQUENCY
INTERVAL
Suboxone, Kratom, Herion,
Suboxone-CAUSE-Cephalalgia
disgusted, amazed, irritated
more than, a, few of
I, me, mine, my
Im glad, turn out bad, weird
ointment, tablet, pill, film
smoke, inject, snort, sniff
Itching, blisters, flushing, shaking
hands, difficulty breathing
DOSAGE: <AMT><UNIT>
(e.g. 5mg, 2-3 tabs)
FREQ: <AMT><FREQ_IND><PERIOD>
(e.g. 5 times a week)
INTERVAL: <PERIOD_IND><PERIOD>
(e.g. several years)
Smarter Data Generated with Ontologies

23. Thank You
Visit Us @ knoesis.org
Follow us @ facebook.com/Kno.e.sis
One example of commercial applications: ezdi.com
with additional background at http://knoesis.org/amit/hcls

24. Ohio Center of Excellence in Knowledge-enabled Computing -
An Ohio Center of Excellence in BioHealth Innovation
Wright State University