The document proposes an ensemble technique that combines different unimodal recommender systems to generate recommendations based on multimodal user interactions. It uses matrix factorization and Bayesian personalized ranking as unimodal recommenders. An algorithm is presented that averages the scores from each unimodal recommender to produce a final recommendation list. The experiments show the proposed ensemble approach achieves better results than the baseline unimodal recommenders in terms of MAP and Precision metrics.

1. Multimodal Interactions in Recommender Systems: An EnsemblingApproach
ARTHUR FORTES E MARCELO G. MANZATO

2. Summary
•Introduction
•Unimodal Recommender Systems
•Proposal
•Experiments and Results
•Conclusions
•Future Works
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH

3. Introduction
•Increase in data on the Web (users, items, reviews)
•The traditional recommendation engines consist in acquiring the preferences:
•Implicit Feedback
•Explicit Feedback
•Literature reports a lack of techniques which integrate different types of user feedback into a generic model
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 3

4. Introduction
•The proposal uses:
•Ensemble technique
•Multimodal interactions
•Unimodal algorithms
•To generate a more accurate list of recommendations optimized for the user
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 4
Figure 1. Interactionsbyusers

5. Unimodal Recommender Systems
•Each unimodalrecommender uses a single or a simple subset of types of user feedback to generate a list of items
•The set of unimodalrecommenders that are used by our algorithm are:
•Matrix Factorization (MF)
•BPR MF (Bayesian Personalized Ranking)
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 5

6. Unimodal Recommender Systems
•Matrix Factorization (MF)
•Matrix factorization techniques allow the discovery of latent features underlying the interactions between users and items
•BPR MF
•The BPR MF approach consists of providing personalized ranking of items to a user according only to implicit feedback (e.g. navigation, clicks, etc.)
•Considers positive and negative feedback
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 6

7. Proposal
•We propose a framework capable of generating recommendations based on multimodal user interactions (Positive and Negative)
•Post-processing step which combines classifications generated by different unimodalrecommenders
•Interactions used:
-Ratings assigned by users (1-5)
-Tags assigned (0 | 1)
-History View (0 | 1)
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 7

8. Proposal
Figure 2. Framework
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 8

9. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 9
Figura 3. Proposedalgorithm

10. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 10
Figura 3. Proposedalgorithm
UserItem Score
5231 7.423
5 8 7.212
5123 6.232
....
20 33 6.823
20 8 6.112
20 54 5.232
...
N 1 8.423
N89 3.212
N 23 6.232

11. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 11
Figura 3. Proposedalgorithm
UserItem Score
5231 7.423
5 8 7.212
5123 6.232
....
20 33 6.823
20 8 6.112
20 54 5.232
...
N 1 8.423
N89 3.212
N 23 6.232

12. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 12
Figura 3. Proposedalgorithm
R(u, t)
5 231 7.4
5 8 7.2
5123 6.2
...
R(u, h)
5 8 8.7
5 325 8.2
552 7.8
...
R(u, r)
5 8 5
5 25 4.5
5572 4
...
R(u, partial)
U I S
58 ?
...

13. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 13
Figura 3. Proposedalgorithm
UserItem Score
5231 7.423
5 8 7.212
5123 6.232
AvgR = 6.9556

14. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 14
Figura 3. Proposedalgorithm
R(u, t)
5 231 7.4
...
R(u, h)
5 231 8.7
...
R(u, r)
5231 5
...
R(u, partial)
U I S
58 8.7
...
Avg(5, t) = 6.4
Avg(5, h) = 5.8
Avg(5, r) = 3

15. Proposal
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 15
Figura 3. Proposedalgorithm

16. Experiments and Results
•Database:
HetRecMovielens2k:
800,000 ratings
10,000 tags
2,113 users
10,197 Movies
•EvaluationMetrics: Map@N; Prec@N; With: 10 crossfoldvalidationandAll-but-oneProtocol
•Recommendationlibrary:
MyMediaLite3.10
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 16

17. Experiments and Results
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 17
Figura 4. ComparativeMAP@N

18. Experiments and Results MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 18
Figura 5. ComparativePrec@N

19. Conclusions
•MAP has a tendency for higher values as the number of returned items increases while Precision has the opposite effect
•MAP only considers the relevant items and their positions in the ranking
•In Precision the order of items is irrelevant, the more items are filtered to the user, the more false positives may also be returned
•Explicit feedback achieved the worst results using matrix factorization.
•Using the proposed ensemble algorithm, we achieved better results than the baselines
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 19

20. Future Works
•MachineLearning Methods
•Extension of the learning algorithm BPR MF
•Group-basedtechniquesfor recommendation
•Usingclusteringalgorithms
MULTIMODAL INTERACTIONS IN RECOMMENDER SYSTEMS: AN ENSEMBLING APPROACH 20

21. Multimodal Interactions in Recommender Systems: An EnsemblingApproach
ARTHUR FORTES E MARCELO G. MANZATO
{FORTES; MMANZATO}@ICMC.USP.BR