The document explores the application of statistical language models in recommender systems, highlighting their relationship with information retrieval and filtering techniques. It presents improvements to relevance-based language models for collaborative filtering and discusses various smoothing methods and priors that enhance recommendation quality. Results from experiments on the MovieLens 100K dataset demonstrate the competitive performance of these models, along with proposed future research directions involving context-aware and hybrid recommendations.

1. Exploring Statistical Language Models
for Recommender Systems
Daniel Valcarce
daniel.valcarce@udc.es – http://www.irlab.org
Information Retrieval Lab, Computer Science Department, University of A Coruña
Information Retrieval (IR)
Goal Retrieve relevant documents according to the information
need of a user.
Examples Search engines.
Methods They can be based on:
Vector Vector Space Model.
Matrix factorisation Latent Semantic Indexing.
Probabilistic modelling Language Models.
Information Fitering (IF)
Goal Select relevant items from an information stream for a
given user.
Examples spam filters, recommender systems.
Methods Some Collaborative Filtering methods are:
Vector Pairwise similarities (cosine, Pearson, etc.).
Matrix factorisation SVD, NMF.
Probabilistic modelling LDA.
Overview
• Information Filtering (IF) and Information Retrieval (IR) are two sibling fields.
• Statistical Language Models are a successful technique in IR → Explore how to apply them to recommendation.
• We start by improving the current adaptation of Relevance-Based Language Models to Collaborative Filtering [1].
Relevance-Based Language Models
IR RecSys
Query Target user
Document Neighbour
Term Item
RM2 : p(i|Ru) ∝ p(i)
j∈Iu v∈Vu
p(i|v) p(v)
p(i)
p(j|v)
• Iu is the set of items rated by the user u.
• Vu is the set of neighbours of the user u.
• p(i|u) is computed smoothing the maximum likelihood es-
timate.
• p(i) and p(v) are the item and user priors.
Smoothing methods
Smoothing deals with data sparsity and plays a similar role to
the IDF using a background model: p(i|C) = v∈U rv,i
j∈I, v∈U rv,j
[3].
Jelinek-Mercer
(JM)
pλ(i|u) = (1 − λ)
ru,i
j∈Iu
ru,j
+ λ p(i|C)
Dirichlet Priors
(DP)
pµ(i|u) =
ru,i + µ p(i|C)
µ + j∈Iu
ru,j
Absolute
Discounting
(AD)
pδ(i|u) =
max(ru,i − δ, 0) + δ |Iu| p(i|C)
j∈Iu
ru,j
Priors
Priors provide a principled way of introducing knowledge into
the recommender [2].
Uniform (U) Linear (L)
User
prior
pU (u) =
1
|U|
pL(u) = i∈Iu
ru,i
v∈U j∈Iv
rv,j
Item
prior
pU (i) =
1
|I|
pL(i) = u∈Ui
ru,i
j∈I v∈Uj
rv,j
Experiments on MovieLens 100k
Algorithm nDCG@10 Gini@10 MSI@10
SVD 0.0946 0.0109 14.6129
SVD++ 0.1113 0.0126 14.9574
NNCosNgbr 0.1771 0.0344 16.8222
UIR-Item 0.2188 0.0124 5.2337
PureSVD 0.3595 0.1364 11.8841
RM2-JM 0.3175 0.0232 9.1087
RM2-DP 0.3274 0.0251 9.2181
RM2-AD 0.3296 0.0256 9.2409
RM2-AD-L-U 0.3423 0.0264 9.2004
Research directions
• Some techniques developed for solving IR problems
can be effectively applied to recommendation.
• Probabilistic models from IR are competitive recom-
mendation algorithms although there is still room for
improvements.
• Language Models provide an interpretable and prin-
cipled way of generate recommendations.
• Using different priors [2] or clustering algorithms for
the neighbourhoods [1] can improve RM2.
• We envision as future work the development of
context-aware and hybrid recommendations under
the Language Modelling.
Bibliography
[1] J. Parapar, A. Bellogín, P. Castells, and A. Bar-
reiro. Relevance-Based Language Modelling for Recom-
mender Systems. Information Processing & Management,
49(4):966–980, 2013.
[2] D. Valcarce, J. Parapar, and A. Barreiro. A Study of Priors
for Relevance-Based Language Modelling of Recommender
Systems. In RecSys ’15. ACM, 2015.
[3] D. Valcarce, J. Parapar, and A. Barreiro. A Study of
Smoothing Methods for Relevance-Based Language Mod-
elling of Recommender Systems. In ECIR ’15, volume 9022,
pages 346–351. Springer, 2015.
RecSys 2015, 9th ACM Conference on Recommender Systems. 16 - 20 September, 2015, Vienna, Austria.