Presented at the Expert Panel Discussion: The Future of Telehealth Technology at National Telehealth Conference, 10 Oct 2017, Cincinnati: http://www.nationaltelehealthconference.com This is an abridged version of an invited talk: https://youtu.be/wDi1mLLyxuc

1. Augmented Personalized Health:
How Smart Data with IoTs and AI is about to change
Healthcare
Panel on The Future of Telehealth Technology
at the National Telehealth Conference, 10 October, 2017; Cincinnati OH
Prof. Amit Sheth
LexisNexis Ohio Eminent Scholar; Executive Director, Kno.e.sis
Wright State University
Background: http://bit.ly/k-APH, http://bit.ly/kAsthma, http://j.mp/PARCtalk,

2. 2
• Traditional Healthcare
• Healthcare: Then and Now
• Augmented Personalized Health for health care
of the future, and associated technical
challenges
• kHealth: Three ongoing applications
Outline

3. 3
Traditional Healthcare
How are you feeling
today?I am not feeling well
since yesterday
afternoon.

4. 4
Healthcare: Then and Now
Episodic Continuous

5. 5
Healthcare: Then and Now
Disease focussed Beyond medical intervention:
Lifestyle change/holistic,
Wellness, Quality of Life

6. 6
Healthcare: Then and Now
Clinic centric Patient centric
(Anywhere the patient is)

7. 7
Healthcare: Then and Now
Clinician control Patient empowered

8. 8
Healthcare: Then and Now
Limited data 360 degree multimodal
● Personal-Public-Population
● Physical-Cyber-Social big data
driven

9. The Patient of the
Future
MIT Technology Review,
2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/9
Patients are increasingly taking control of own health
Patient Generated Health Data (PGHD) are “health-related data created, recorded, or gathered by or from
patients (or family members or other caregivers) to help address a health concern. PGHD include, but are not
limited to health history, treatment history, biometric data, symptoms, and lifestyle choices.”
Office of the National Coordinator for Health Information Technology (ONC).

10. Google:Verily Life Sciences
https://blog.verily.com/2017/04/introducing-verily-study-watch.html

11. AIM: Doctor in a self-driving Car
[Image: courtesy Artefac, https://flipboard.com/@flipboard/-who-needs-a-hospital-when-this-self-dri/f-4204314e11%2Ffastcodesign.com t]

12. 12
Healthcare as we know it is in the process of going through a
massive change - from episodic to continuous, from disease
focused to wellness and quality of life focused, from clinic
centric to anywhere a patient is, from clinician controlled to
patient empowered, and from being driven by limited data to
360-degree, multimodal personal-public-population physical-
cyber- social big data driven.
Converting big data into smart data through contextual and
personalized processing such that patient and clinician can
make better decisions and take timely actions for Augmented
Personalized Health
Future Health Care

13. 13image: https://cdn1.tnwcdn.com/wp-content/blogs.dir/1/files/2013/12/augmented-reality-doctors-lab.jpg
Augmented Personalized Health
Augmented Personalized Healthcare (APH) is expected
to enhance healthcare by personalizing the use of all
relevant
Physical, Cyber, and Social data obtained from wearables,
sensors and Internet of Things, mobile applications,
electronic medical records, web-based information,
and social media for better health for an individual.
Data include traditional clinical data, PGHD and public health data, as well as
environmental and social data that could impact an individual’s health.

14. 14
Providing actionable information in a
timely manner is crucial to avoid
information overload or fatigue
Sleep data
Community
dataPersonal
Schedule Activity data
Personal
health
records
Data Overload for Patients/health aficionados

15. 15
Challenges in deriving actionable insights from Sensor Data
According to Forbes, the wearables market exceeded $2 billion in
2015, 3 billion in 2016 and will be over 4 billion in 2017.
Leading to vast volume of healthcare data: some key issues
● Sensor reliability and Quality
● Sensor data Heterogeneity
● Contextual Interpretation and Abstraction
● Personalized Health and Health Objective
https://www.linkedin.com/pulse/digital-health-why-doctors-should-care-doug-hart

16. 17
Is sensor data useful for Health Decision making?

17. Hyperthyroidism
Elevated
Blood
Pressure
Systolic blood pressure of 150
mmHg
“150”
..
.
..
.
Too much data does not help make timely decisions =>
Contextual Interpretation and Sensor Data Abstraction

18. Personalized Health and Objectives: one size does not fit all
Millions of people - > one treatment
Wearable and Sensor data

19. 22
• Asthma Management in Children
• Bariatrics Surgery for Obese Adults
• Pain Management
kHealth Personalized Digital Health Initiative

20. kHealth Asthma Management

21. 24
kHealth: Health Signal Processing Architecture
Personal
level Signals
Public level
Signals
Population
level Signals
Domain
Knowledge
Risk Model
Events from Social
streams
Take Medication before
going to work
Avoid going out in the
evening due to high pollen
levels
Contact doctor
Analysis
Personalized
Actionable
Information
Data Acquisition
& aggregation

22. 25
25Asthma Domain Knowledge
Domain
Knowledge
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-
agonist ;
*consider referral to specialist
Asthma Control and Actionable Information
Asthma Control
Daily Medication
choices for
stating therapy Not Well Controlled
Poor Controlled
Severity Level of
Asthma
Intermittent Asthma
Mild Persistent Asthma
Moderate Persistent Asthma
Severe Persistent Asthma
Recommended Action Recommended Action Recommended Action
SABA prn
Low dose ICS
Medium ICS alone or
with LABA/Montelukast
High dose
LABA/Montelukast
Medium ICS
Medium ICS +
LABA/ Montelukast or
High Dose ICS
Need specialist care
Medium ICS
Medium ICS +
LABA/ Montelukast or
High Dose ICS
Need specialist care

23. 26
Foobot – for monitoring
environmental air quality
Wheezometer – for
monitoring
wheezing sounds
Can I reduce my asthma
attacks at night?
What are the triggers?
What is the wheezing level?
What is the propensity
toward asthma?
What is the exposure level
over a day?
Commute to work
Decision Support for Doctors and Patients: A Scenario
Luminosity
CO
level
CO in gush
during day
time Actionable
Information
Personal level
Signals
Public level
Signals
Population
level Signals
What is the air quality indoors?
Close the window at home
during day to avoid CO2
inflow, to avoid asthma
attacks at night

24. 27
k-Health Dashboard: A Platform to Visually Analyse to find
Correlations (e.g., Patient Symptoms and Personalized Data)
Multimodal Data Streams & Anonymised Patient Data Visualized for Correlation Analysis
interpreted in with the help of knowledge graph (relevant medical knowledge)

25. 28
Activity limitation observed with high pollen activity

26. 29
Low exhaled nitric oxide observed with absence of coughing

27. 30
Medication use possibly leading to decreasing exhaled nitric oxide

28. 31
Activity Limitation is likely related to high exhaled nitric oxide

29. 33
33
Risk
assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population
level Signals
GREEN -- Well Controlled
YELLOW – Not well controlled
Red -- poor controlled
How controlled is my asthma?
Patient Health Score (Diagnostic)

30. 34
34
Risk
assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population
level Signals
How vulnerable* is my control level today?
Patient Health Score (Prognostic)

31. kHealth Bariatrics
The purpose of our research is to determine if monitoring Bariatric patient’s pre- and postoperative
compliance with active and passive sensors can bolster bariatric patient’s progress and lessen
weight recidivism

32. ● 500 million people all over the world are obesity
● 36% of the adults in the United States suffer from obesity
● 65% of the world’s population lives in countries where the occurrence of death
due to overweight and obesity is higher than being underweight
Obesity

33. ● Chances of regaining weight as stomach can still expand after surgery
● Continuous monitoring of the patients by the surgeon is very essential
Bariatric Surgery

34. Challenges Post-Bariatric Surgery
● Patient acceptance and active participation involving continuous
monitoring of the patient
● Cost and reimbursement models
● Challenging research in understanding of variety of data over long period

35. A system that can
● monitor the patient continuously and remotely
● identify non-compliance before and after surgery
● nudge/assist for better compliance for improved outcomes and reduce
recidivism
Post-Bariatric Surgery Solution

36. kHealth Post-Bariatric Surgery Proposed Method
Aggregate the data collected from the sensors, questionnaires and use artificial
intelligence techniques to:
● analyse and predict the deviations that could cause the post surgical
complications and,
● serve as an assistant leading to better patient-compliance and outcomes

37. kHealth Bariatrics: Kit

38. 42
42
How do we solve problems with real world complexity, gather vast
amount of data, diverse knowledge……. and come up with
intelligent decisions that works for an individual at a given time?
next: a pedagogical take

39. Semantic Perceptual Cognitive computing

40. Interplay between Semantic, Cognitive and Perceptual Computing (SC, CC and PC) with examples
More here- Video, Slides
Semantic Perceptual Cognitive computing in two use cases:
Asthma and Traffic Management

41. 45
Thank you
Thank you, and please visit us at
http://knoesis.org
For more information on kHealth, please visit us at
http://knoesis.org/projects/khealth
Cognitive
Computing
Semantic
Computing
Perceptual
Computing
Contributors and collaborators for this talk:
Pramod
Anantharam
Cory
Henson
Dr. T.K.
Prasad
Sanjaya
Wijeratne
Utkarshani
Jaimini

42. Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University