This document presents a knowledge-aware food recommender system that uses holistic user models. It aims to address limitations of content-based and collaborative filtering approaches. The proposed system uses a profiler to build comprehensive user profiles incorporating demographics, behaviors, health data and domain knowledge. Recipes are then filtered and ranked using this user information and food knowledge rules. An evaluation with 200 MTurk participants found users with health goals preferred recipes recommended by the holistic model over popular recipes. The study provides initial evidence the approach can better support healthy eating goals.

1. Towards a Knowledge-aware
Food Recommender System
Exploiting Holistic User Models
Cataldo Musto, Christoph Trattner, Alain Starke, Giovanni Semeraro
ACM UMAP 2020

2. Food Recommender Systems
?
Goal: to identify the
most suitable recipe for a target user
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

3. Background
• Content-based approaches are not effective
– Based on similarity between recipes
– Issue: over-specialization
• Collaborative approaches are not effective
– Most of the users prefer popular (and popular recipes are unhealthy)
– Not good, especially for health-aware food recsys
• State-of-the-art techniques have a limited understanding of user
characteristics and goals
– Limited exploitation of personal features (BMI, mood, physical activity)
and constraints (dietary preferences)
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

4. Contribution
• A Knowledge-aware Food recommender
System based on Holistic User Models
– Knowledge-based Recommender Systems
avoid both over-specialization and popularity
bias
– Holistic User Models (HUM) provide a
comprehensive representation of the users that
encodes under-investigated features
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

5. Methodology

6. Knowledge-aware Food RecSys
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

7. Knowledge-aware Food RecSys
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

8. Profiler
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
Demographics
Affects
Behavioral Data
Health Data
Domain Knowledge
Our user profiling
strategy relies on
the principles of
Holistic User
Modeling (*)
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
(*) Musto, C., Semeraro, G., Lovascio, C., de Gemmis,
M., Lops, P. A framework for Holistic User Modeling
merging heterogeneous digital footprints. In Adjunct
Publication – UMAP 2018.

9. Profiler
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
Mood
Level of Physical Activity
Cooking Experience
Food Requirements, Amount of
Sleep, Stress Level, Weight (BMI)
Gender, Sex, AgeDemographics
Affects
Behavioral Data
Health Data
Domain Knowledge
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

10. Knowledge-aware Food RecSys
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

11. Filter
• Carries out some preliminary filtering of the
available recipes
– Based on the characteristics of the user
– E.g., remove recipes containing meat for
vegetarian users, or recipes containing
lactose for lactose-intolerant users
– Produces a set of candidate recipes
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

12. Knowledge-aware RecSys
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

13. Ranker
• Exploits characteristics of the users and
background knowledge about food to rank the
available recipes
• Given a user u, for each recipe r the following scoring
formula is applied
– holistic(r,u) =
popScore(r) ∗ knowledge(r,u)
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

14. Ranker
• Exploits characteristics of the users and
background knowledge about food to rank the
available recipes
• Given a user u, for each recipe r the following scoring
formula is applied
– holistic(r,u) =
popScore(r) ∗ knowledge(r,u)
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
?

15. Food Knowledge
• The method exploits a knowledge base encoding
commonsense and background knowledge about food
consumption and healthy food habits
– e.g., overweight users → recipes with lower calories
– e.g., much physical activities → recipes with more proteins
and carbs
– e.g., bad mood → recipes with high sugar
• Characteristics of the user are matched to the available rules
– The application of the rule influences the score of the
recipe (and, in turn, the ranking)
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

16. Use Case
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
?
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020

17. Step 1: popularity-based score
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
?
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
6.8
6.4
6.1 7.5
7.5
6.6

18. Step 2: holistic user modeling
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
?
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
low medium male good BMI>25
6.8 6.1 7.5
7.5
6.6
6.4

19. Step 3: knowledge-based ranking
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
?
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
↘ 5.8
↗ 7.5
↗ 7.1 ↘ 7.4
↘ 5.5
↗ 6.8
low medium male good BMI>25

20. By exploiting a more
comprehensive
representation of the
users and background
knowledge about food,
more satisfying
recommendations can
be obtained
Step 4: recipe recommendation
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD RECOMMENDER
SYSTEM EXPLOITING HOLISTICUSER MODELS – ACM UMAP 2020
C.MUSTO, C.TRATTNER, A.STARKE, G.SEMERARO – TOWARDS A KNOWLEDGE-AWARE FOOD
RECOMMENDER SYSTEM EXPLOITING HOLISTIC USER MODELS – ACM UMAP 2020
!
:)low medium male good BMI>25
↗ 7.5

21. Experiment –
User study
N = 200 Mturk Participants with 99% HIT rate

22. Procedure
• Step 1:
build user model

23. • Step 1: build a user model
• Step 2: each user was asked to choose between 3
pairs of recipes:
a 1st course; a 2nd course, and a dessert
User study on our holistic user model

24. Evaluation of recipe pairs
Holistic vs Popular

25. Evaluation of recipe pairs
Holistic vs Popular

26. Results

27. Personal factors (e.g., BMI, age) hardly affected choices
for either holistic or popular recipes
No effects: For 2nd courses:
BMI
Age
Mood
Web
use
Cooking
freq.
Gender
(male)
Holistic
recipes
–Gender
(male)
Predicting user choices for
holistic recommendations

28. Users who indicated to have based on health preferred
holistic recipes, others users (goals) popular ones
No effects:
Weight
Reason
Health
Reason Taste
Reason
Weight
Goals
Holistic
Desserts
Holistic 1st
courses
–
–
Ease to
cook
Predicting user choices for
holistic recommendations

29. Why were recipes chosen?
Recipe features vs user factors
Holistic recipes were more likely to be
chosen if they had healthy features

30. Why were recipes chosen?
Recipe features vs user factors

31. Conclusion

32. A first step towards developing a knowledge-aware
RecSys to support healthy food goals
• Choices for holistic recipes are associated with healthy food goals
• Users with taste-related food goals stick to popular alternatives
• Proposed follow-up: How can holistic user modelling
support healthy maintenance behaviors?
Take-aways (read the paper ☺)