This document discusses deep learning recommender systems from prototypes to production. It provides an overview of modern recommender systems and how deep learning techniques like neural item embeddings, similarity search, and experimentation can improve recommender systems. The key points are: (1) Deep learning allows extracting features from different data sources and generating accurate user/item representations; (2) Neural item embeddings like word2vec learn vector representations of items to find similar items; (3) Similarity search techniques like ANN enable efficient nearest neighbor search in large embedding spaces; (4) Experimentation through offline/online testing and A/B testing is important for evaluating models and improving recommendations.
2. Outline
● About us
● Modern Recommender Systems
● Deep Learning Recommender Systems
● Neural Item Embeddings
● Similarity Search
● Proving value through Experimentation
● From POC to PRD
● Summary
3. About us
ILIA IVANOV
Data Scientist
ilia.ivanov@olx.com
CRISTIAN MARTINEZ
Lead Data Scientist
cristian.martinez@olx.com
7. Amazon
Amazon researchers
found that using neural
networks to generate
movie recommendations
worked much better when
they sorted the input data
chronologically and used
it to predict future movie
preferences over a short
(one- to two-week)
period.
Source: https://www.amazon.science/the-history-of-amazons-recommendation-algorithm
9. Content-based
Deep content-based music recommendation (Oord et al, 2013)
● Convolutional neural nets
● Key ideas:
○ Extract latent representations
of songs from audio signals
○ Train CNN network to generate
embeddings of songs
○ The input of the CNN is a
time-frequency representation
of the audio
10. Neural Item Embeddings
E-commerce in Your Inbox (Grbovic et al, 2015)
● Inspired by word2vec architecture
● Key ideas:
○ Generate embeddings of products in a
word2vec fashion
○ “Words” are products and “sentences”
are purchase sequences of a user
○ Learn product embeddings using
skip-gram
11. Wide & Deep
Wide & Deep Learning for Recommender Systems (Cheng et al, 2016)
● Wide and deep feed-forward architectures
● Key ideas:
○ Jointly train two neural
networks.
○ Deep neural network
trained with embeddings.
○ (Wide) Linear model with
feature transformations
for generic recommender
systems with sparse inputs.
12. Session-based
Session-based recommendations with RNNs (Hidasi et al, 2016)
● Recurrent neural network architectures
● Key ideas:
○ Train using sequence of items in a session
○ Predict: next item in the session
○ Pairwise (ranking) loss functions
■ Bayesian personalised ranking (BPR)
13. Graph-based
Graph Convolutional Neural Networks for Web-Scale Recommender
Systems (Ying et al, 2018)
● GCN architectures
● Key ideas:
○ Combines efficient random
walks and graph
convolutions to
generate embeddings
of nodes (e.g. items)
○ Improve quality of item
embedding by extracting
Information from its
neighbor nodes.
14. Benefits of using DNNs
● Extract deep features directly from content
○ e.g. CNNs for images, LSTMs for text
● Allow us to combine information extracted from
different input sources
● Generate more accurate representations of users
and items
● Allow us to use custom loss functions
17. 2018
Deep neural network marketplace
recommenders in online experiments
Combine content-based features with user
behaviors to solve the cold start challenge of
collaborative filtering.
2019
How we use item2vec to recommend similar
products
Avito’s item2vec implementation.
+30% contacts from i2i recommendations.
+20% contacts from personal recommendations.
2015
E-commerce in Your Inbox: Product
Recommendations at Scale
Use word2vec algorithm to learn embedding
representations of products.
2016
Meta-Prod2Vec: Product Embeddings Using
Side-Information for Recommendation
Prod2vec + meta-data about the item.
22. References
● E-commerce in Your Inbox: Product Recommendations at
Scale (Grbovic et al, 2016)
● Meta-Prod2Vec: Product Embeddings Using
Side-Information for Recommendation (Vasile et al, 2016)
● Deep neural network marketplace recommenders in
online experiments (Eide et al, 2018)
● Как мы используем item2vec для рекомендаций похожих
товаров Avito’s item2vec (Russian language)
24. Search
Problem: find the nearest neighbors among millions of item embeddings
Solution: approximate nearest neighbor (ANN)
Implementations:
● FAISS (FB)
● Annoy (Spotify)
● NMSLIB
● Open Distro for ES