Slides of our presentation at the WSDM 2019 Conference. Paper available at https://arxiv.org/abs/1811.09975. Code available at https://github.com/noveens/svae_cf. Abstract: Variational autoencoders were proven successful in domains such as computer vision and speech processing. Their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention in the current literature. In this work, we propose a model which extends variational autoencoders by exploiting the rich information present in the past preference history. We introduce a recurrent version of the VAE, where instead of passing a subset of the whole history regardless of temporal dependencies, we rather pass the consumption sequence subset through a recurrent neural network. At each time-step of the RNN, the sequence is fed through a series of fully-connected layers, the output of which models the probability distribution of the most likely future preferences. We show that handling temporal information is crucial for improving the accuracy of the VAE: In fact, our model beats the current state-of-the-art by valuable margins because of its ability to capture temporal dependencies among the user-consumption sequence using the recurrent encoder still keeping the fundamentals of variational autoencoders intact.

1. Sequential Variational Autoencoders for
Collaborative Filtering
Noveen Sachdeva*, Giuseppe Manco, Ettore Ritacco, Vikram Pudi
*supported by SIGIR Student Travel Grant and Flipkart

2. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

3. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

4. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

5. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

6. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

7. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

8. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

9. Context
Recommendation as Preference Reconstruction
Input
Encoder
Code
Output
Decoder

10. Variational Autoencoders for Recommender
Systems
• Elegant formulation as an encoder-decoder problem
• Robust Mathematical Model based on variational approximation of the
data likelihood
• Encoder/decoder as probabilistic functions implemented through NNs

11. • Elegant formulation as an encoder-decoder problem
• Robust Mathematical Model based on variational approximation of the
data likelihood
• Encoder/decoder as probabilistic functions implemented through NNs
• Disregards the sequential nature of preferences
Variational Autoencoders for Recommender
Systems

12. Sequential Variational Autoencoder
• At each step, history is encoded
and exploited to decode the
next preference
• Encoding shaped via a Recurrent
Nework
…
Input
Encoder
Code
Output
Decoder

13. Time
Impact
Input
Encoder
Code
Output
Decoder

14. Time
Impact
Input
Encoder
Code
Output
Decoder

15. Time
Impact
Input
Encoder
Code
Output
Decoder

16. Time
Impact
Input
Encoder
Code
Output
Decoder

17. Time
Impact
Input
Encoder
Code
Output
Decoder

18. Time
Impact
Input
Encoder
Code
Output
Decoder

20. Evaluation
• Global accuracy
Model
Movielens Netflix
NDCG Recall Precision NDCG Recall Precision
@10 @100 @10 @100 @10 @100 @10 @100 @10 @100 @10 @100
POP 4.24 11.30 3.41 22.15 3.98 3.14 3.95 7.61 1.29 12.02 4.26 3.95
BPR∗ 8.51 16.66 5.20 29.38 7.27 4.51 8.94 12.68 2.63 19.45 8.45 5.89
RVAE 7.76 14.02 4.11 23.82 6.39 3.96 4.19 9.46 1.87 17.00 3.87 4.30
FPMC 5.65 12.06 3.46 23.10 4.77 3.46 11.01 13.55 3.49 20.82 10.12 6.04
CASER 16.32 27.55 11.35 46.01 13.15 6.45 16.46 19.79 6.33 28.33 14.38 7.54
MVAE 11.69 23.01 9.12 41.43 9.02 5.39 16.15 22.19 7.47 32.72 13.94 8.31
SVAE 17.81 29.93 12.48 49.09 14.40 6.93 24.64 26.77 8.93 35.58 21.93 10.55
MVAE∗
27.82 39.79 17.46 59.70 23.01 9.59 35.41 37.70 13.50 47.22 31.39 15.20

21. Evaluation
• Accuracy v/History Length

22. Evaluation
• Accuracy v/Prediction Lookahead

23. Thank you
giuseppe.manco@icar.cnr.it
@beman
Questions?