More Related Content


More from James Hendler(20)

Recently uploaded(20)


Enhancing Precision Wellness with Personal Health Knowledge Graphs

  1. Enhancing Precision Wellness with Personal Health Knowledge Graphs Professor James Hendler Director, Rensselaer Institute for Data Exploration and Analytics
  2. PHKG: Outside the clinical… • If you are not sick, you see your doctor rarely or never – This is true even if you have a propensity for a chronic condition • If you have a chronic condition, you see your doctor periodically – Depends on disease, but typically relatively frequent until stable, then less often • For most people, the biggest impact on health is their behaviors – Every single day!
  3. Nota bene • In this talk I am going to focus on the “knowledge level” which means I’m leaving out a bunch of huge issues: – Usability – Personal decision making – Incentive mechanisms – Advice taking and Habit formation – … • (Could be good panel topics)
  4. 75% Social Context, Behaviors Impact on population health status 20% Medical Care 5% Genetic Accessing a vast amount of untapped data could have a great impact on our health - yet it exists outside medical systems SOURCE: Barbara J. Sowada, A Call to Be Whole: The Fundamentals of Health Care Reform, July 30, 2003, Praeger. 6 Terabytes EMR, EHR, claim systems Consumer-contributed data from non- medical sources: wearables, social media activity, public records 1100 Terabytes Generated per lifetime 0.4 Terabytes Genetic research, personalized medicine, clinical trials IBM Watson // ©2015 IBM Corporation
  5. Precision Health/Wellness Precision Medicine Precision Health/ Wellness Lifestyle and environmental impacts on the individual including behaviors Biomedical understanding of the individual based primarily on genomics Individual’s health
  6. HEALTH EMPOWERMENT by Putting the right information into the hands of patients and clinicians when they need it the most ANALYTICS Using data for hypothesis formation and testing LEARNING & Improving knowledge continuously SEMANTICS Integrating health & medical knowledge from heterogeneous sources HEALS is a member of the IBM AI Horizons Network Rensselaer-IBM HEALS project
  7. Example: Diabetes management In the United States In China Pre-COVID diabetes was 7th largest cause of death in US (now 8th)
  8. Type 2 Diabetes: Prevention and Treatment • Prevalence • In 2015, 30.3 million Americans, or 9.4% of the population, had diabetes. (90- 95% of these people have T2D) • 24% of them didn’t know they had it. • Pre-diabetes • More than 1 out of 3 Americans have prediabetes. Of those with prediabetes, 90% don’t know they have it. • Cost • The average medical expenditure for people with diagnosed diabetes is about $16,750 per year, of which about $9,600 is due to diabetes. “Statistics about Diabetes”, American Diabetes Association. “Prediabetes: Your Chance to Prevent Type 2 Diabetes”, CDC. “Economic Costs of Diabetes in the U.S. in 2017”. American Diabetes Association, Diabetes Care May 2018, 41 (5) 917-928; DOI: 10.2337/dci18-0007
  9. Problem Objectives Technical Challenge Relevant knowledge is heterogeneous Personal information must be aligned with disease-related information • Disease progression & risk factors • Treatment guidelines • Nutritional & physical activity habits and preferences • Other lifestyle constraints Resolving inconsistencies between disparate sources Relevant knowledge is dynamic Data sources (e.g., personal context, social media, web pages) are constantly changing and must be reconciled Keeping knowledge current Patients have unique needs Individual’s information needs are dependent on their specific micro- to macro-level contexts Deriving personalized insights Raw insights are not interpretable or actionable A system must educate on good practices, support planning for improved health, and track and evaluate performance Delivering explainable recommendations Personalized Information
  10. I'm not losing weight, what else can I do? I see that you're walking about 5,000 steps/day, which is below the American Heart Association's recommendation. However, I can see from your calendar that you don't have a lot of free time to focus on physical activity. Maybe we should we talk about your diet even thought I don't have a lot of information about your eating habits? System recognizes concept of "weight" that is tied to diet and activity. System analyzes both activity and diet data in the personal KG (sourced from Apple HealthKit) for the last month: • Activity data: Compares Ed's activity with the recommended guidelines (10,000 steps per day AHA recommendation), and determines that he is not getting enough activity from walking (Ed averages only 5,124 steps per day). • Calendar data: Use iOS EventKit and finds Ed has very little free time during the day. • Diet data: Ed doesn't log his food, so system needs to ask him about his dietary habits. Sure. I see that you usually go to Applebee's for lunch during the week, what do you usually order? However, the system recognizes that he spends lunch time at Applebee's every day (sourcing location data from iOS CoreLocation API matched with restaurant listings from the Google Maps API ). Conversation Back-End System ED ED
  11. I usually get a battered fish sandwich and a side of fries. Ok, that’s 981 calories and 43g of fat for lunch. Guidelines recommend that you should be consuming 2086 calories/day so your lunch is 47% of your daily calories. For more information, click here. System retrieves nutrition facts about Ed's "average" lunch from the nutrition KG and populates Ed's personal KG with the self-reported data about his diet habits. System calculates recommended daily caloric intake for Ed based on: • Ed's height 5'10'', current weight 200lb, age 57, caloric intake etc. from Ed's personal knowledge graph • Guidelines on recommended caloric intake • System computes recommended caloric intake from the Mifflin-St. Jeor equation (for BMR) & fat intake from Ed's personal KG , compares to his data What is a healthier option? Next time try a 6-oz top sirloin, baked potato, and fire- grilled veggies for lunch. This only has 540 calories and fewer than 22 grams of fat. I also found other restaurants near you with healthy options. Would you like more information? System retrieves the Applebee's menu with nutrition information from the Nutrition KG and recommends healthier food substitutes at the same restaurant that are in the similar categories to Ed's usual lunch. ED ED Conversation Back-End System
  12. Note: Work in progress
  13. I'm not losing weight, what else can I do? It looks like you have been active so let's look at your diet. Can you explain what the graph means? It looks like you usually have a big dinner and snack at night. You also don't eat lunch most days. Also, you tend to have more calories over weekends. System uses Watson NLU to recognize concept of "losing weight" that is tied to diet and activity. Then, system analyzes both activity and diet data stored in the personal KG (sourced from Apple HealthKit) for the last month: • Activity data: Compare Jessica's activity with the recommended guidelines (10,000 steps per day AHA recommendation), and determines that she is almost getting sufficient activity 9,356 steps per day) from walking. • Diet data: Jessica logs food in MyFitnessPal, which syncs with Apple Health. The system compares her recommended caloric intake (1700 kCal) and her average caloric intake. On weekdays her caloric intake is (2200 kCal), and weekends she consumes (3000 kCal)System mined pattern from personal KG: she has a large gap between breakfast and dinner and she typically consumes 2/3 of her daily calories at dinner in addition to snacks at night. She doesn't eat lunch and also eats more on the weekends. JE Conversation Back-End System JE
  14. Why is this a problem? I usually am not hungry during the day so I just eat when I get home from work. According to WebMD, The longer the gap between dinner and the previous meal or snack, the larger the dinner. People who eat lightly at night end up eating fewer calories and grams of fat overall than people who eat big dinners and nighttime snacks. For more, click here. Do I really eat a lot of fat at night? According to AHA guidelines, You should be consuming 51 grams of fat/day. On average, you eat 70 grams at night. Here is a breakdown of your average fat content for breakfast, lunch, dinner, and snacks. For more, click here. System extracted explanation from WebMD, by searching for <late night snacks and big dinners>. System calculates recommended fat intake for Jessica based on: • Her height, current weight, caloric intake etc. from Jessica's personal KG • AHA guidelines on recommended fat intake • System computes recommended caloric intake from the Mifflin-St. Jeor equation (for BMR) & fat intake from Jessica's personal KG , compares to her data Conversation Back-End System JE JE
  15. What info goes where? • General <-> Specific – If your biological parent had diabetes you are more susceptible to the disease – My mother had diabetes
  16. What info goes where? • General <-> Specific – GENERAL • Search “blood sugar 103”
  17. What info goes where? • General <-> Specific – Specific: I’m an overweight 60+ yr old man with a family history of diabetes and my blood sugar is 103 • Better answer: some worry
  18. What info goes where? • General <-> Specific – Specific: I’m an overweight 60+ yr old man with a family history of diabetes and my blood sugar is 103 but last year it was 115 • Better answer: Keep it up!
  19. Integration of sources needed +
  20. Knowledge Graph for Personal Health PERSONAL DISEASE DIET ACTIVITIES Web sources & online forums Personal health data AI/ML Technologies - Data-mining / KDD - NLP - Semantic Search - Semantic Data Integration - ”rule based” reasoning
  21. Health records for large cohort Clinical Record Mining for chronic condition information (structured and unstructured) Image Mining for Tissue/Cell level information Test results yield biochemical level information Mined Knowledge Graph (aggregate) Personal Health Knowledge Graph(s) Online Behavior(s) Genetic Information (i.e. genetic details of patient cohorts) Putting it all together
  22. Mined Knowledge Graph (aggregate) Online Behavior(s) Putting it all together Apps using the PHKG (which lives in my space) can mediate information exchange Personal Health Knowledge Graph(s)
  23. Challenges • Separation/control of information – Power of big data vs. Individual rights • Integration is power – Learn from what is happening across a population • Separation is privacy preserving – Who controls what is shared?
  24. Challenges • Reasoning Challenges – Combining Learning and Reasoning • Learning is powerful – Eating Broccoli can boost the immune system • Reasoning is needed for the individual – I don’t like broccoli
  25. Challenges • Some other issues – Data Provenance • Did my doctor say that or did it come from the Web? – Uncertainty • I think I had a few pieces of chocolate this afternoon • (different devices mentioned this AM) – Temporal issues / “abduction” • My blood sugar over time is a better indicator than my instantaneous / can we explain change over time? …
  26. Challenges • Eternal issues will impact if we can bring these things to the public – Privacy and security issues as discussed – Public policy issues (an even longer talk than this one) – Health care in most parts of the world is big business and very unresponsive to change • Your doctor’s primary job is to fix you when you are broken – Worldwide investment in biomedical research (esp pharma) drastically outweight investiment in health care reseach
  27. Conclusions • Healthcare and lifestyle information is more powerful when set in a personal context – More likely to be affective – More likely to be followed • Knowledge graphs can power linking and thus learning – But information may live in silos • The solution requires an amalgam of approaches

Editor's Notes

  1. Feb 2008 Rehearsal
  2. J.M. McGinnis et al., “The Case for More Active Policy Attention to Health Promotion,” Health Affairs 21, no. 2 (2002):78–93
  3. HEALS is an acronym that stands for Health Empowerment by Analytics, Learning & Semantics. We seek to create health empowerment by putting the right information into the hands of the patients and clinicians who need it the most. We will do this by combining the best that mathematical, cognitive and semantic technologies have to offer in terms of SEMANTICally integrating health and medical knowledge from heterogeneous sources, Improving that knowledge continuously through LEARNING And ANALYZING that data to formulate and test hypotheses
  4. Source:
  5. Show how the demos they will see today will demonstrate different aspects of the technical challenges we are addressing… HEALS 2 characterize disease in the face of changing information provide context for overwhelming amounts of complex information (personalized population analysis) leverage structured domain knowledge to help filter/prioritize/hypothesize
  6. Personal background information: Sociodemographic information (e.g., age, gender, family structure, profession), medical history (comorbidities), personality characteristics Health related parameters: Step counts, heart rate, blood pressure, food intake, liquid intake, activity data, stress data, BMI, weight -- temporal and geo-location data Social: Cultural influences, infer through data or asking explicitly (food restrictions), social environment , social media use, people they like to spend time with, people they trust/respect Resources/barriers: Free time or time-constraints, dietary restrictions, resources/restrictions that they have (e.g., do they have a car? how much do they make?), skills that they have (e.g., literacy, languages, education) Preferences and willingness: Times they prefer to engage in different activities, activities they prefer to engage in, locations/communities they prefer or frequent, what they prefer to eat, willingness to try new things (food, activities, plans, etc.) Motivations: Concerns that they have, goals Knowledge and experiences: Knows what they have tried in the past regarding health behaviors (e.g., diets, exercise), what has worked  and not worked for them, past experiences, stories they heard about other people's experiences