Keynote talk at personal health knowledge graph

1. Enhancing Precision Wellness with
Personal Health Knowledge Graphs
Professor James Hendler
Director, Rensselaer Institute for Data Exploration and Analytics

2. PHKG: Outside the clinical…
• If you are not sick, you see your doctor rarely or never
– This is true even if you have a propensity for a chronic
condition
• If you have a chronic condition, you see your doctor
periodically
– Depends on disease, but typically relatively frequent until
stable, then less often
• For most people, the biggest impact on health is their
behaviors
– Every single day!

3. Nota bene
• In this talk I am going to focus on the “knowledge level”
which means I’m leaving out a bunch of huge issues:
– Usability
– Personal decision making
– Incentive mechanisms
– Advice taking and Habit formation
– …
• (Could be good panel topics)

4. 75%
Social Context, Behaviors
Impact on population health status
20%
Medical Care
5%
Genetic
Accessing a vast amount of untapped data could have a great
impact on our health - yet it exists outside medical systems
SOURCE: Barbara J. Sowada, A Call to Be Whole:
The Fundamentals of Health Care Reform, July 30, 2003, Praeger.
6 Terabytes
EMR, EHR, claim systems
Consumer-contributed data from non-
medical sources: wearables, social media
activity, public records
1100 Terabytes
Generated per lifetime
0.4 Terabytes
Genetic research, personalized
medicine, clinical trials
IBM Watson // ©2015 IBM Corporation

5. Precision Health/Wellness
Precision Medicine Precision Health/
Wellness
Lifestyle and
environmental
impacts on the
individual
including
behaviors
Biomedical
understanding of
the individual
based primarily on
genomics
Individual’s
health

6. HEALTH
EMPOWERMENT by
Putting the right information into the hands of patients
and clinicians when they need it the most
ANALYTICS Using data for hypothesis formation and testing
LEARNING & Improving knowledge continuously
SEMANTICS
Integrating health & medical knowledge from
heterogeneous sources
HEALS is a member of the IBM AI Horizons Network
Rensselaer-IBM HEALS project

7. Example: Diabetes management
In the United States In China
Pre-COVID diabetes was 7th largest cause of death in US (now 8th)

8. Type 2 Diabetes: Prevention and
Treatment
• Prevalence
• In 2015, 30.3 million Americans, or 9.4% of the population, had diabetes. (90-
95% of these people have T2D)
• 24% of them didn’t know they had it.
• Pre-diabetes
• More than 1 out of 3 Americans have prediabetes. Of those with prediabetes,
90% don’t know they have it.
• Cost
• The average medical expenditure for people with diagnosed diabetes is about
$16,750 per year, of which about $9,600 is due to diabetes.
“Statistics about Diabetes”, American Diabetes Association. http://www.diabetes.org/diabetes-basics/statistics/
“Prediabetes: Your Chance to Prevent Type 2 Diabetes”, CDC. https://www.cdc.gov/diabetes/basics/prediabetes.html
“Economic Costs of Diabetes in the U.S. in 2017”. American Diabetes Association, Diabetes Care May 2018, 41 (5) 917-928; DOI: 10.2337/dci18-0007

9. Problem Objectives Technical Challenge
Relevant knowledge is
heterogeneous
Personal information must be aligned with disease-related information
• Disease progression & risk factors
• Treatment guidelines
• Nutritional & physical activity habits and preferences
• Other lifestyle constraints
Resolving inconsistencies
between disparate sources
Relevant knowledge is
dynamic
Data sources (e.g., personal context, social media, web pages) are
constantly changing and must be reconciled
Keeping knowledge current
Patients have unique
needs
Individual’s information needs are dependent on their
specific micro- to macro-level contexts
Deriving personalized
insights
Raw insights are not
interpretable or actionable
A system must educate on good practices, support planning for improved
health, and track and evaluate performance
Delivering explainable
recommendations
Personalized Information

11. I'm not losing weight, what else can I do?
I see that you're walking about 5,000 steps/day,
which is below the American Heart Association's
recommendation. However, I can see from your
calendar that you don't have a lot of free time to
focus on physical activity. Maybe we should we talk
about your diet even thought I don't have a lot of
information about your eating habits?
System recognizes concept of "weight" that is
tied to diet and activity.
System analyzes both activity and diet data in
the personal KG (sourced from Apple
HealthKit) for the last month:
• Activity data: Compares Ed's activity with
the recommended guidelines (10,000 steps
per day AHA recommendation), and
determines that he is not getting enough
activity from walking (Ed averages only
5,124 steps per day).
• Calendar data: Use iOS EventKit and finds
Ed has very little free time during the day.
• Diet data: Ed doesn't log his food, so system
needs to ask him about his dietary habits.
Sure.
I see that you usually go to Applebee's for lunch during
the week, what do you usually order?
However, the system recognizes that he spends
lunch time at Applebee's every day (sourcing
location data from iOS CoreLocation API matched
with restaurant listings from the Google Maps API
).
Conversation Back-End System
ED
ED

12. I usually get a battered fish sandwich and a side of
fries.
Ok, that’s 981 calories and 43g of fat for lunch. Guidelines
recommend that you should be consuming 2086
calories/day so your lunch is 47% of your daily calories.
For more information, click here.
System retrieves nutrition facts about Ed's "average"
lunch from the nutrition KG and populates Ed's
personal KG with the self-reported data about his
diet habits.
System calculates recommended daily caloric intake
for Ed based on:
• Ed's height 5'10'', current weight 200lb, age 57,
caloric intake etc. from Ed's personal knowledge
graph
• Guidelines on recommended caloric intake
• System computes recommended caloric intake
from the Mifflin-St. Jeor equation (for BMR) & fat
intake from Ed's personal KG , compares to his
data
What is a healthier option?
Next time try a 6-oz top sirloin, baked potato, and fire-
grilled veggies for lunch. This only has 540 calories and
fewer than 22 grams of fat. I also found other restaurants
near you with healthy options. Would you like more
information?
System retrieves the Applebee's menu with nutrition
information from the Nutrition KG and recommends
healthier food substitutes at the same restaurant that
are in the similar categories to Ed's usual lunch.
ED
ED
Conversation Back-End System

13. Note: Work in progress

14. I'm not losing weight, what else can I do?
It looks like you have been active so let's look at your
diet.
Can you explain what the graph means?
It looks like you usually have a big dinner and snack at
night. You also don't eat lunch most days. Also, you
tend to have more calories over weekends.
System uses Watson NLU to recognize concept
of "losing weight" that is tied to diet and activity.
Then, system analyzes both activity and diet
data stored in the personal KG (sourced from
Apple HealthKit) for the last month:
• Activity data: Compare Jessica's activity with
the recommended guidelines (10,000 steps
per day AHA recommendation), and
determines that she is almost getting
sufficient activity 9,356 steps per day) from
walking.
• Diet data: Jessica logs food in MyFitnessPal,
which syncs with Apple Health. The system
compares her recommended caloric intake
(1700 kCal) and her average caloric intake.
On weekdays her caloric intake is (2200
kCal), and weekends she consumes (3000
kCal)System mined pattern from personal KG: she
has a large gap between breakfast and dinner
and she typically consumes 2/3 of her daily
calories at dinner in addition to snacks at night.
She doesn't eat lunch and also eats more on
the weekends.
JE
Conversation Back-End System
JE

15. Why is this a problem? I usually am not hungry
during the day so I just eat when I get home from
work.
According to WebMD,
The longer the gap between dinner and the
previous meal or snack, the larger the dinner.
People who eat lightly at night end up eating fewer
calories and grams of fat overall than people who
eat big dinners and nighttime snacks. For more,
click here.
Do I really eat a lot of fat at night?
According to AHA guidelines,
You should be consuming 51 grams of fat/day. On
average, you eat 70 grams at night. Here is a
breakdown of your average fat content for breakfast,
lunch, dinner, and snacks. For more, click here.
System extracted explanation from WebMD,
by searching for <late night snacks and big
dinners>.
System calculates recommended fat intake
for Jessica based on:
• Her height, current weight, caloric intake
etc. from Jessica's personal KG
• AHA guidelines on recommended fat
intake
• System computes recommended caloric
intake from the Mifflin-St. Jeor equation
(for BMR) & fat intake from Jessica's
personal KG , compares to her data
Conversation Back-End
System
JE
JE

17. What info goes where?
• General <-> Specific
– If your biological parent had diabetes you are
more susceptible to the disease
– My mother had diabetes

18. What info goes where?
• General <-> Specific
– GENERAL
• Search “blood sugar 103”

19. What info goes where?
• General <-> Specific
– Specific: I’m an overweight 60+ yr old man with a
family history of diabetes and my blood sugar is
103
• Better answer: some worry

20. What info goes where?
• General <-> Specific
– Specific: I’m an overweight 60+ yr old man with a
family history of diabetes and my blood sugar is
103 but last year it was 115
• Better answer: Keep it up!

21. Integration of sources needed
+

22. Knowledge Graph for Personal Health
PERSONAL
DISEASE
DIET
ACTIVITIES
Web sources & online forums
Personal health data
AI/ML Technologies
- Data-mining /
KDD
- NLP
- Semantic Search
- Semantic Data
Integration
- ”rule based”
reasoning

23. Health
records for
large cohort
Clinical Record
Mining for
chronic condition
information
(structured and
unstructured)
Image Mining for
Tissue/Cell level
information
Test results yield
biochemical level
information
Mined
Knowledge
Graph
(aggregate)
Personal Health
Knowledge Graph(s)
Online
Behavior(s)
Genetic
Information (i.e.
genetic details of
patient cohorts)
Putting it all together

24. Mined
Knowledge
Graph
(aggregate)
Online
Behavior(s)
Putting it all together
Apps using the PHKG (which lives in my space) can mediate information exchange
Personal Health
Knowledge Graph(s)

25. Challenges
• Separation/control of information
– Power of big data vs. Individual rights
• Integration is power
– Learn from what is happening across a population
• Separation is privacy preserving
– Who controls what is shared?

26. Challenges
• Reasoning Challenges
– Combining Learning and Reasoning
• Learning is powerful
– Eating Broccoli can boost the immune system
• Reasoning is needed for the individual
– I don’t like broccoli

27. Challenges
• Some other issues
– Data Provenance
• Did my doctor say that or did it come from the Web?
– Uncertainty
• I think I had a few pieces of chocolate this afternoon
• (different devices mentioned this AM)
– Temporal issues / “abduction”
• My blood sugar over time is a better indicator than my
instantaneous / can we explain change over time?
…

28. Challenges
• Eternal issues will impact if we can bring these things to
the public
– Privacy and security issues as discussed
– Public policy issues (an even longer talk than this one)
– Health care in most parts of the world is big business and
very unresponsive to change
• Your doctor’s primary job is to fix you when you are broken
– Worldwide investment in biomedical research (esp
pharma) drastically outweight investiment in health care
reseach

29. Conclusions
• Healthcare and lifestyle information is more
powerful when set in a personal context
– More likely to be affective
– More likely to be followed
• Knowledge graphs can power linking and thus
learning
– But information may live in silos
• The solution requires an amalgam of approaches