NAVER Engineering, profile picture
Uploaded byNAVER Engineering
2,188 views

Recent advances in deep recommender systems

The document discusses recent advancements in deep learning-based recommender systems, highlighting the basics and methodologies such as collaborative filtering and matrix factorization. It explains various deep learning techniques used for product, movie, and music recommendations, along with challenges in modeling user-item interactions. Additionally, it explores specific models like AutoRec and Prod2Vec that utilize deep learning for improving recommendation accuracy.

Embed presentation

Downloaded 147 times
Recent Advances in Recommender Systems using Deep Learning August 20, 2019 Sungkyunkwan University (SKKU) Prof. Jongwuk Lee
2 Recommender Systems Basics
What are Recommender Systems? 3 Items Recommendation products, movies, news, …
Search vs. Recommendation  How can we help users get access to relevant data?  Pull mode (search engines)  Users take initiative.  Ad-hoc information need  Push mode (recommender systems)  Systems take initiative.  Stable information need or a system has user’s information need. 4
Product Recommendation 5 Amazon product recommendation
Movie & Music Recommendation 6 Netflix movie recommendation Spotify music recommendation
News Recommendation 7 Naver AiRs Kakao Rubics
Various Recommendations 8 Social/Friend recommendation Restaurant recommendation Tag recommendationApp recommendation
What is Collaborative Filtering?  Given a target user, Alice, find a set of users whose preference patterns are similar to that of the target user.  Predict a list of items that Alice will be likely to prefer. 9 Target user: Alice ① Inferring Alice’s preference ② Finding a set of users with similar preference for Alice ③ Recommending a list of items that a user group prefers Top-N Recommendation
User-Item Rating Matrix  A user-item rating matrix R of the target user, Alice, and other users is given:  R: a user-item rating matrix (𝒎 × 𝒏 matrix)  Determine whether She would like or dislike movies, which has not seen yet. 10 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … … … … … … Alice
Latent Factor Models  How to model user-item interactions?  U: latent user matrix (𝒎 × 𝒌 matrix)  Each user is represented by a latent vector (1 × 𝑘 vector).  V: latent item matrix (𝒏 × 𝒌 matrix)  Each item is represented by a latent vector (1 × 𝑘 vector). 11 User-item interaction 𝒇 𝑼, 𝑽 = ? …… ……
Factorizing Two Latent Matrices  The user-item rating matrix R can be approximated as a linear combination of two latent matrices U and V.  R: user-item rating matrix (𝑚 × 𝑛 matrix)  U: latent user matrix (𝑚 × 𝑘 matrix)  V: latent item matrix (𝑛 × 𝑘 matrix)  𝑘: # of latent features 12 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … k features kfeatures R U VT … … … … … Yehuda Koren, “Factorization meets the neighborhood: a multifaceted collaborative filtering model,” KDD 2008
Limitation of Existing Models  Existing models are mainly based on a linear user-item interaction.  However, the user-item interaction may be non-linear and non-trivial. 13 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5… … k features kfeatures R U VT … … … … …
14 Deep Recommender Systems
Statistics of RecSys Models using DNNs  The number increases exponentially in the last five years.  SIGIR, WWW, RecSys, KDD, AAAI, WSDM, NIPS, … 15 Shuai Zhang et al., “Deep Learning based Recommender System: A Survey and New Perspectives,” 2017
Categories of RecSys Models using DNNs 16 Deep Learning based Recommender System Model using Single DL Technique Integrate DL with Traditional RS Deep Composite Model Recommend Rely Solely on DL MLP AE CNN RNN DSSM RBM NADE GAN Loosely Coupled Model Tightly Coupled Model Integration ModelNeural Network Model
Categories of RecSys Models using DNNs 17 Deep Learning based Recommender System Model using Single DL Technique Integrate DL with Traditional RS Deep Composite Model Recommend Rely Solely on DL MLP AE CNN RNN DSSM RBM NADE GAN Loosely Coupled Model Tightly Coupled Model Integration ModelNeural Network Model
AutoRec: Autoencoder-based Model  For each item, reconstruct rating vectors.  Observed ratings are only used to update the model. 18 2 1 3… 2 1 3… …𝒉(𝒓) 𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … Suvash Sedhain et al. “AutoRec: Autoencoders Meet Collaborative Filtering,” WWW 2015
Denoising Autoencoder (DAE)  Learn to reconstruct a user’s favorite set ෤𝒓 of items from randomly sampled subsets, i.e., denoising autoencoder. 19 3 2… 3 3 2… …𝒉(𝒓) ෤𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ Denoising input 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … Yao Wu et al., “Collaborative Denoising Auto-Encoders for Top-N Recommender Systems,” WSDM 2016
Collaborative Denoising Autoencoder  Learn to reconstruct a user’s favorite set ෤𝒓 of items from randomly sampled subsets, i.e., denoising autoencoder.  Train for all users by shared variables for items and a specific variable for each user, i.e., 𝒌 × 𝟏 vector. 20 3 2… 3 3 2… …𝒉(𝒓) ෤𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ 1 0 0… 𝑽 𝒌×𝟏 One-hot vector for user Denoising input 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … …
Generalized Matrix Factorization (GMF)  Embeddings are used to learn latent user and item features.  Input: one-hot feature vector for user u and item i  Output: predicted score ŷui  Element-wise product is same as the existing MF model. 21 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Element-wise product Fully connected w/o bias Layer … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … … Xiangnan He et al., “Neural Collaborative Filtering,” WWW 2017
Step 1: Embedding Users and Items  User embedding  Represent a latent feature for each user.  Item embedding  Similarly, it represents a latent feature for each item. 22 𝟎 𝟏 𝟎 … 𝟎 × 𝟎. 𝟔 𝟎. 𝟖 𝟎. 𝟔 𝟎. 𝟒 𝟎. 𝟐 𝟎. 𝟓 ⋯ 𝟎. 𝟔 𝟏. 𝟐 𝟎. 𝟐 ⋮ ⋱ ⋮ 𝟎. 𝟕 𝟎. 𝟓 ⋯ 𝟎. 𝟑 = 𝟎. 𝟔 𝟎. 𝟒 … 𝟏. 𝟐 𝟏 × 𝒎 vector 𝒎 × 𝒌 matrix 𝟏 × 𝒌 vector 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … …
Step 1: Embedding Users and Items 23 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … R U VT … … … … … …
Step 2: Element-wise Product  This is same as the linear combination of U and VT. 24 Element-wise product Layer 𝟎. 𝟕𝟐 … 𝟎. 𝟑𝟔 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … … … … … … 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 R U VT
Step 3: Passing Fully Connected Layer 25 ෝ𝒚 𝒖𝒊 Fully connected w/o bias 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … R … … … … … If the weight is equal, it is same as the existing MF model. 𝑹 = 𝒘(𝑼⨂𝑽 𝑻) Element-wise product Layer 𝟎. 𝟕𝟐 … 𝟎. 𝟑𝟔 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 U VT
MLP-based Matrix Factorization  Instead of element-wise product, concatenate latent user and item vectors. 26 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Concatenating two latent vectorsLayer 1 Layer X ……. Fully connected w/o bias … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … …
MLP-based Matrix Factorization  Learning non-trivial interactions between users and items 27 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Concatenating two latent vectorsLayer 1 Layer X ……. Fully connected w/o bias … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑
Fusing Two Solutions  Neural collaborative filtering: Both use GMF and MLP layers. 28 0 0 1 … 00 1 0 … 0 MF user vector MLP item vectorMLP user vector MF item vector MLP Layer 1 MLP Layer X ……. Element-wise product GMF Layer ෝ𝒚 𝒖𝒊 Concatenation Fully connected w/o bias NeuMF Layer Concatenation ……
Example: Product Recommendation 29 https://www.bloter.net/archives/288812
Distributional Hypothesis  “You shall know a word by the company it keeps.” by J.R. Firth (1957)  Words with similar contexts share similar meanings.  One of the most successful ideas of modern NLP  What is Tejuino ?  A cup of Tejuino is on the table.  A woman likes Tejuino.  Tejuino makes you drunk.  I usually drink Tejuino every morning. 30
Prod2Vec using Word Embeddings  How to perform product embedding?  Adopt the skip-gram model for products.  Input: i-th product purchased by the user  Output: the other product purchased by the user 31 A set of words  A set of products purchased by the user Mihajlo Grbovic et al., “E-commerce in Your Inbox: Product Recommendations at Scale,” KDD 2015
Prod2Vec using Word Embeddings  Imagine that the existing user-item matrix.  Word  Movie, A set of words  User  The window size is ignored. 32 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … i-th movie watched by the user Projection Softmax Transform The other movie watched by the user
Possible Models of Prod2Vec 33 i-th product purchased by the user Projection Softmax Transform The other product purchased by the user i-th product purchased by the user Projection + Averaging Softmax Transform user products except for the i-th product purchased by the user Prod2Vec Skip-gram model User embedding + Prod2Vec
34 About Time (2013) 드라마/판타지/성장 어떠한 순간을 다시 살게 된다면, 과연 완벽한 사랑을 이룰 수 있을까? 모태솔로 팀은 성인이 된 날, 아비저로부터 놀랄만한 가문의 비밀을 듣게 된다. 바로 사긴을 되돌릴 수 있는 능력이 있다는 것! 여자친구를 만든다는 꿈을 이루기 위해 런던으로 간 팀은 메리에게 첫눈에 반하게 되는데.. Silver Linings Playbook (2012) 드라마/코미디 Secret Life of Walter Mitty, The (2013) 판타지/드라마 Perks of Being a Wallflower, The (2012) 드라마/성장 The Theory of Everything(2014) 드라마/로맨스 What If (2013) 로맨스/코미디 Man Up (2015) 드라마/로맨스 Love, Rosie (2014) 로맨스/코미디 Two Night Stand (2014) 로맨스/코미디 GloVe Skip-gram
Mulan (1998) 애니메이션 동양의 분위기를 살린 디즈니의 역작! 파씨 가문의 외동딸 뮬란은 선머슴 같은 성격 때문에 중매를 볼 때마다 퇴짜를 맞는다. 때마침 훈족의 침입으로 징집명령이 떨어지고 늙은 아버지를 대신하여 남장을 하고 나서는데.. Jungle Book, The (1967) 애니메이션 Antz (1998) 애니메이션 Lady and the Tramp (1955) 애니메이션 Peter Pan (1953) 애니메이션 Thumbelina (1994) 애니메이션 A Dinosaur's Story (1993) 애니메이션 Quest for Camelot (1998) 애니메이션 Return to Never Land (2002) 애니메이션 35 GloVe Skip-gram
Machine, The (2013) 판타지/SF(로봇) 인간과 로봇 그 경계가 사라진다! Signal, The (2014) 스릴러/SF (컴퓨터) Zero Theorem, The (2013) 판타지/드라마(컴퓨터) Autómata (2014) 스릴러/액션(로봇) Cargo (2009) 스릴러/미스터리(우주) 신 냉전시대에 인간의 뇌 데이터를 바탕으로 탄생한 살인로봇 머신 에이바는 점차 인간의 감정을 느껴가고, 그녀를 주축으로 머신들은 인간과의 최후의 전쟁을 선포하는데… the east(2013) 스릴러/액션(스파이) Signal, The (2014) 스릴러/SF (컴퓨터) Autómata (2014) 스릴러/액션(로봇) Cargo (2009) 스릴러/미스터리(우주) 36 GloVe Skip-gram
Session-based Recommendation  Sequential behavior (global information)  Last click (local information) 37 Session input Recommendation
GRU4Rec: RNN-based Model 38 Item2Vec RNN Layer Balazs Hidasi et al., “Session-based Recommendations with Recurrent Neural Networks”, RecSys Workshop 2015
NARM: Attention-based Model  Recommend the next item in a given session.  Combine global and local information.  They are represented by RNNs. 39 Jing Li et al., “Neural Attentive Session-based Recommendation,” CIKM 2017
Step 1: Global Encoder in NARM  Capturing user’s sequential behavior  ℎ 𝑡 = 1 − 𝑧𝑡 ℎ 𝑡−1 + 𝑧𝑡 ෠ℎ 𝑡  𝑧𝑡 = 𝜎(𝑊𝑧 𝑥 𝑡 + 𝑈𝑧ℎ 𝑡−1) where 𝑧𝑡 is update gate  ෠ℎ 𝑡 = tanh[𝑊𝑥 𝑡 + 𝑈 𝑟𝑡 ⊙ ℎ 𝑡−1 ]  𝑟𝑡 = 𝜎(𝑊𝑟 𝑥 𝑡 + 𝑈𝑟ℎ 𝑡−1) 40
Step 2: Local Encoder in NARM  Capturing user’s main purpose  𝛼 𝑡𝑗 = 𝑞 𝒉 𝒕, ℎ𝑗 , where 𝒉 𝒕 is a latent vector for the last item.  𝐶𝑡 𝑙 = σ 𝑗=1 𝑡 𝛼 𝑡𝑗ℎ𝑗  𝑞 is an attention scoring function for ℎ 𝑡 and ℎ𝑗.  𝑞 ℎ 𝑡, ℎ𝑗 = 𝑣 𝑇 𝜎(𝐴1ℎ 𝑡 + 𝐴2ℎ𝑗) 41
Step 3: Decoder in NARM  Concatenated vector 𝑐𝑡 = 𝑐𝑡 𝑔 ; 𝑐𝑡 𝑙 = [ℎ 𝑡 𝑔 ; σ 𝑗−1 𝑡 𝛼 𝑡𝑗ℎ 𝑡 𝑙 ]  Use an alternative bi-linear similarity function.  𝑆𝑖 = 𝑒𝑚𝑏𝑖 𝑇 𝐵𝑐𝑡 where 𝐵 is a 𝐷 × |𝐻| matrix.  |𝐷| is the dimension of each item. 42
Combining Attention and Memory  𝒎 𝒔 represents the average vector of items.  𝑚 𝑠 = 1 𝑡 σ𝑖=1 𝑡 𝑥𝑖  𝒎 𝒕 is the vector for last item. 43 Qiao Liu et al., “Short-Term Attention/Memory Priority Model for Session-based Recommendation,” KDD 2018
STAMP: Attention/Memory Priority  𝒎 𝒂 is sum of multiplication between coefficient and embedding vector.  𝑚 𝑎 = σ𝑖=1 𝑡 𝛼𝑖 𝑥𝑖  Attention coefficient: 𝛼𝑖 = 𝑊0 𝜎(𝑊1 𝑥𝑖 + 𝑊2 𝒎 𝒔 + 𝑊3 𝒙 𝒕 + 𝑏) 44
MMCF: Multimodal Collaborative Filtering for Automatic Playlist Continuation RecSys Challenge 2018 Team ‘hello world!’ (2nd place), main track Hojin Yang, Yoonki Jeong, Minjin Choi, and Jongwuk Lee Sungkyunkwan University, Republic of Korea
Automatic Playlist Continuation  Million Playlist Dataset (MPD) 46 Playlist title Tracks in the playlist Metadata of tracks (artist, album)
Challenge Set 47 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Few tracks in the first part Many tracks in the first part Many tracks in the random position
Challenge Set 48 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 No tracks in the playlist How to deal with an edge case?
Challenge Set 49 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Scarce information How to treat playlists with scarce information?
Challenge Set 50 1 2 3 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Various types of Input How to deal with various types of input?
Overview of the Proposed Model  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 51
Overview of the Proposed Model  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 52
Autoencoder-based Model 53 1 0 1 1 0 1Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder  Learn a latent representation of a given playlist consisting of a set of tracks 0.9 0.01 0.78 0.9 0. 6 0.8 ✔ Top-1 Recommendation Output
Denoising Autoencoder 54 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder 1 0 1 1 0 1  Training with denoising  Some positive input values are corrupted (set to zero). denoising
Denoising Autoencoder 55  Training with denoising  Some positive input values are corrupted (set to zero). How to utilize the metadata such as artists and albums? 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder 1 0 1 1 0 1 denoising
Utilizing Metadata 56 1 0 1 1 0 1 0 1 1 0 0.9 0.01 0.78 0.9 0. 6 0.8 0.2 0.98 0.9 0.6 Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder  Concatenate an artist vector corresponding to the track vector in the playlist.
 Randomly choose either the playlist or its artists as input. 57 Training Strategy: Hide-and-Seek
Training Strategy: Hide-and-Seek  Randomly choose either the playlist or its artists as input. 58
Training Strategy: Hide-and-Seek  Randomly choose either the playlist or its artists as input. 59
CharCNN for Playlist Titles  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 60
Word-level CNN for NLP  Effective for capturing spatial locality of a sequence of texts 61 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filter (3 by k ) 2.2 2.3 -1.3 0.9 max pooling Conv layer 2.3 Feature much convolution k-dimension embedding
Word-level CNN for NLP  Effective for capturing spatial locality of a sequence of texts 62 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filters (3 by k ) convolution 2.2 2.3 -1.3 0.9 max pooling 2.3 Feature much k-dimension embedding Conv layer 2.2 2.3 -1.3 0.9 Conv layers 2.2 2.3 -1.3 0.9 1.2 2.4 -1.1 0.4 max pooling 2.3 1.2 2.4 Feature vector convolution
CharCNN for Playlist Titles  Playlist titles are represented by a short text, implying an abstract description of a playlist.  Use character-level embedding. 63 Conv layers Feature vector
Combining Two Models  Simplest method: 𝑤𝒊𝒕𝒆𝒎 = 0.5 and 𝑤𝒕𝒊𝒕𝒍𝒆 = 0.5 64
Combining Two Models  The accuracy of the AE highly relies on the number of tracks within a playlist.  Dynamic: Set weights according to the number of items. 65 Items Playlist Title Chill songs 0.7 0.4 0.9 0.1 0.2 0.1 0.2 0.3 0.7 0.1 0.6 0.4 0.7 0.2 0.2 AE CNN 𝑤_𝑖𝑡𝑒𝑚 = 5 𝑤_𝑡𝑖𝑡𝑙𝑒 = 1
RecSys Challenge 2018 66
Recent Progress in Our Lab  “Dual Neural Personalized Ranking,” WWW 2019  “Characterization and Early Detection of Evergreen News Articles,” ECML/PKDD 2019 (To appear)  “Collaborative Distillation for Top-N Recommendation,” ICDM 2019 (To appear) 67
Q&A  https://jongwuklee.weebly.com/ 68
SR-GNN using Graph Neural Nets  Each session graph is proceeded one by one. Node vectors can be obtained through a gated graph neural network.  Each session is represented as the combination of the global preference and current interests of this session using an attention net. 69 Shu Wu et al., “Session-based Recommendation with Graph Neural Networks,” AAAI 2019
RecSys Challenge 2018 70

More Related Content

PPTX
Recommendation Systems
byRobin Reni
 
PPTX
Recommender Systems
byGirish Khanzode
 
PDF
Recommender system algorithm and architecture
byLiang Xiang
 
PPTX
Recommender system
byNilotpal Pramanik
 
PPTX
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
 
PDF
Overview of recommender system
byStanley Wang
 
PPTX
Recommender system introduction
byLiang Xiang
 
PDF
An introduction to Recommender Systems
byDavid Zibriczky
 
Recommendation Systems
byRobin Reni
 
Recommender Systems
byGirish Khanzode
 
Recommender system algorithm and architecture
byLiang Xiang
 
Recommender system
byNilotpal Pramanik
 
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
 
Overview of recommender system
byStanley Wang
 
Recommender system introduction
byLiang Xiang
 
An introduction to Recommender Systems
byDavid Zibriczky
 

What's hot

PDF
Recommendation System Explained
byCrossing Minds
 
PDF
Recommender Systems
byFrancesco Casalegno
 
PDF
Recommender Systems
byCarlos Castillo (ChaTo)
 
PDF
Introduction to Recommendation Systems
byTrieu Nguyen
 
PPTX
Collaborative Filtering Recommendation System
byMilind Gokhale
 
PPTX
Recommender systems using collaborative filtering
byD Yogendra Rao
 
PPT
Recommendation system
byVikrant Arya
 
PPTX
Recommender systems for E-commerce
byAlexander Konduforov
 
PDF
Building a Recommendation Engine - An example of a product recommendation engine
byNYC Predictive Analytics
 
PDF
Boston ML - Architecting Recommender Systems
byJames Kirk
 
PPT
Item Based Collaborative Filtering Recommendation Algorithms
bynextlib
 
PPTX
Recommendation Systems Basics
byJarin Tasnim Khan
 
PPTX
Recommendation System
byAnamta Sayyed
 
PDF
Deep Learning for Recommender Systems
byJustin Basilico
 
PPTX
Recommendation system
byAkshat Thakar
 
KEY
Recommender Engines
byThomas Hess
 
PPTX
Recommendation system
byRishabh Mehta
 
PPTX
Recommender Systems from A to Z – Model Training
byCrossing Minds
 
PDF
Collaborative filtering
byTien-Yang (Aiden) Wu
 
PDF
Deep Learning for Recommender Systems RecSys2017 Tutorial
byAlexandros Karatzoglou
 
Recommendation System Explained
byCrossing Minds
 
Recommender Systems
byFrancesco Casalegno
 
Recommender Systems
byCarlos Castillo (ChaTo)
 
Introduction to Recommendation Systems
byTrieu Nguyen
 
Collaborative Filtering Recommendation System
byMilind Gokhale
 
Recommender systems using collaborative filtering
byD Yogendra Rao
 
Recommendation system
byVikrant Arya
 
Recommender systems for E-commerce
byAlexander Konduforov
 
Building a Recommendation Engine - An example of a product recommendation engine
byNYC Predictive Analytics
 
Boston ML - Architecting Recommender Systems
byJames Kirk
 
Item Based Collaborative Filtering Recommendation Algorithms
bynextlib
 
Recommendation Systems Basics
byJarin Tasnim Khan
 
Recommendation System
byAnamta Sayyed
 
Deep Learning for Recommender Systems
byJustin Basilico
 
Recommendation system
byAkshat Thakar
 
Recommender Engines
byThomas Hess
 
Recommendation system
byRishabh Mehta
 
Recommender Systems from A to Z – Model Training
byCrossing Minds
 
Collaborative filtering
byTien-Yang (Aiden) Wu
 
Deep Learning for Recommender Systems RecSys2017 Tutorial
byAlexandros Karatzoglou
 

Similar to Recent advances in deep recommender systems

PDF
Tutorial: Context In Recommender Systems
byYONG ZHENG
 
PDF
Deep Learning Recommender Systems
byCristian Javier Martinez
 
PDF
Further enhancements of recommender systems using deep learning
byInstitute of Contemporary Sciences
 
PPTX
Recommendation system
byDing Li
 
PPTX
Recommender Systems from A to Z – The Right Dataset
byCrossing Minds
 
PPTX
Neural collaborative filtering-발표
byhyunsung lee
 
PDF
Real-time personalized recommendations using product embeddings
byJakub Macina
 
PDF
Frequently Bought Together Recommendations Based on Embeddings
byDatabricks
 
PPTX
Rokach-GomaxSlides.pptx
byJadna Almeida
 
PDF
Real-world News Recommender Systems
bykib_83
 
PDF
Distributed Representation-based Recommender Systems in E-commerce
byRakuten Group, Inc.
 
PDF
IntroductionRecommenderSystems_Petroni.pdf
byAlphaIssaghaDiallo
 
PDF
Introduction to Recommender System
byWQ Fan
 
PPTX
Rokach-GomaxSlides (1).pptx
byJadna Almeida
 
PDF
Crafting Recommenders: the Shallow and the Deep of it!
bySudeep Das, Ph.D.
 
PPTX
Teacher training material
byVikram Parmar
 
PPTX
HABIB FIGA GUYE {BULE HORA UNIVERSITY}(habibifiga@gmail.com
byHABIB FIGA GUYE
 
PPTX
Олександр Обєдніков “Рекомендательные системы”
byDakiry
 
PDF
Andrew Clegg, Data Scientician & Machine Learning Engine-Driver: "Deep produc...
byDataconomy Media
 
PDF
Introduction to Recommender Systems
byTuri, Inc.
 
Tutorial: Context In Recommender Systems
byYONG ZHENG
 
Deep Learning Recommender Systems
byCristian Javier Martinez
 
Further enhancements of recommender systems using deep learning
byInstitute of Contemporary Sciences
 
Recommendation system
byDing Li
 
Recommender Systems from A to Z – The Right Dataset
byCrossing Minds
 
Neural collaborative filtering-발표
byhyunsung lee
 
Real-time personalized recommendations using product embeddings
byJakub Macina
 
Frequently Bought Together Recommendations Based on Embeddings
byDatabricks
 
Rokach-GomaxSlides.pptx
byJadna Almeida
 
Real-world News Recommender Systems
bykib_83
 
Distributed Representation-based Recommender Systems in E-commerce
byRakuten Group, Inc.
 
IntroductionRecommenderSystems_Petroni.pdf
byAlphaIssaghaDiallo
 
Introduction to Recommender System
byWQ Fan
 
Rokach-GomaxSlides (1).pptx
byJadna Almeida
 
Crafting Recommenders: the Shallow and the Deep of it!
bySudeep Das, Ph.D.
 
Teacher training material
byVikram Parmar
 
HABIB FIGA GUYE {BULE HORA UNIVERSITY}(habibifiga@gmail.com
byHABIB FIGA GUYE
 
Олександр Обєдніков “Рекомендательные системы”
byDakiry
 
Andrew Clegg, Data Scientician & Machine Learning Engine-Driver: "Deep produc...
byDataconomy Media
 
Introduction to Recommender Systems
byTuri, Inc.
 

More from NAVER Engineering

PDF
BPL(Banksalad Product Language) 무야호
byNAVER Engineering
 
PDF
플랫폼 디자이너 없이 디자인 시스템을 구축하는 프로덕트 디자이너의 우당탕탕 고통 연대기
byNAVER Engineering
 
PDF
디자인 시스템에 직방 ZUIX
byNAVER Engineering
 
PDF
서비스 운영을 위한 디자인시스템 프로젝트
byNAVER Engineering
 
PDF
쏘카프레임 구축 배경과 과정
byNAVER Engineering
 
PDF
React vac pattern
byNAVER Engineering
 
PDF
이번 생에 디자인 시스템은 처음이라
byNAVER Engineering
 
PDF
진화하는 디자인 시스템(걸음마 편)
byNAVER Engineering
 
PDF
200820 NAVER TECH CONCERT 15_Code Review is Horse(코드리뷰는 말이야)(feat.Latte)
byNAVER Engineering
 
PDF
날고 있는 여러 비행기 넘나 들며 정비하기
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 05_모르면 손해보는 Android 디버깅/분석 꿀팁 대방출
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 07_신입 iOS 개발자 개발업무 적응기
byNAVER Engineering
 
PDF
200820 NAVER TECH CONCERT 13_네이버에서 오픈 소스 개발을 통해 성장하는 방법
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 09_Case.xcodeproj - 좋은 동료로 거듭나기 위한 노하우
byNAVER Engineering
 
PDF
200820 NAVER TECH CONCERT 11_빠르게 성장하는 슈퍼루키로 거듭나기
byNAVER Engineering
 
PDF
200820 NAVER TECH CONCERT 12_상반기 네이버 인턴을 돌아보며
byNAVER Engineering
 
PDF
200820 NAVER TECH CONCERT 14_야 너두 할 수 있어. 비전공자, COBOL 개발자를 거쳐 네이버에서 FE 개발하게 된...
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 08_성능을 고민하는 슬기로운 개발자 생활
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 10_맥북에서도 아이맥프로에서 빌드하는 것처럼 빌드 속도 빠르게 하기
byNAVER Engineering
 
PDF
200819 NAVER TECH CONCERT 03_화려한 코루틴이 내 앱을 감싸네! 코루틴으로 작성해보는 깔끔한 비동기 코드
byNAVER Engineering
 
BPL(Banksalad Product Language) 무야호
byNAVER Engineering
 
플랫폼 디자이너 없이 디자인 시스템을 구축하는 프로덕트 디자이너의 우당탕탕 고통 연대기
byNAVER Engineering
 
디자인 시스템에 직방 ZUIX
byNAVER Engineering
 
서비스 운영을 위한 디자인시스템 프로젝트
byNAVER Engineering
 
쏘카프레임 구축 배경과 과정
byNAVER Engineering
 
React vac pattern
byNAVER Engineering
 
이번 생에 디자인 시스템은 처음이라
byNAVER Engineering
 
진화하는 디자인 시스템(걸음마 편)
byNAVER Engineering
 
200820 NAVER TECH CONCERT 15_Code Review is Horse(코드리뷰는 말이야)(feat.Latte)
byNAVER Engineering
 
날고 있는 여러 비행기 넘나 들며 정비하기
byNAVER Engineering
 
200819 NAVER TECH CONCERT 05_모르면 손해보는 Android 디버깅/분석 꿀팁 대방출
byNAVER Engineering
 
200819 NAVER TECH CONCERT 07_신입 iOS 개발자 개발업무 적응기
byNAVER Engineering
 
200820 NAVER TECH CONCERT 13_네이버에서 오픈 소스 개발을 통해 성장하는 방법
byNAVER Engineering
 
200819 NAVER TECH CONCERT 09_Case.xcodeproj - 좋은 동료로 거듭나기 위한 노하우
byNAVER Engineering
 
200820 NAVER TECH CONCERT 11_빠르게 성장하는 슈퍼루키로 거듭나기
byNAVER Engineering
 
200820 NAVER TECH CONCERT 12_상반기 네이버 인턴을 돌아보며
byNAVER Engineering
 
200820 NAVER TECH CONCERT 14_야 너두 할 수 있어. 비전공자, COBOL 개발자를 거쳐 네이버에서 FE 개발하게 된...
byNAVER Engineering
 
200819 NAVER TECH CONCERT 08_성능을 고민하는 슬기로운 개발자 생활
byNAVER Engineering
 
200819 NAVER TECH CONCERT 10_맥북에서도 아이맥프로에서 빌드하는 것처럼 빌드 속도 빠르게 하기
byNAVER Engineering
 
200819 NAVER TECH CONCERT 03_화려한 코루틴이 내 앱을 감싸네! 코루틴으로 작성해보는 깔끔한 비동기 코드
byNAVER Engineering
 

Recently uploaded

PPTX
Conversational Agents – Building Intelligent Assistants [Virtual Hands-on Wor...
byUiPathCommunity
 
PDF
10 Things AI-First Apps Do Differently by iProgrammer Solutions
byiProgrammer Solutions Private Limited
 
PPTX
Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]
byUiPathCommunity
 
PDF
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
PDF
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
PPTX
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
PDF
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
PPTX
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
PPTX
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
PDF
Unlocking the Power of Salesforce Architecture: Frameworks for Effective Solu...
byvarsha30tiwari
 
PDF
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
PDF
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
PDF
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
Day 4 - Access, Deployments, and Monitoring - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
Our Digital Tribe_ Cultivating Connection and Growth in Our Slack Community 🌿...
bysanjeetmishra30
 
PDF
Cross-Cultural Agile Development -Challenges and Strategies for Overcoming Them-
byTakashi Makino
 
PPTX
communication-skills-with-technology tools
byJaleto Sunkemo
 
PDF
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
PDF
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 
Conversational Agents – Building Intelligent Assistants [Virtual Hands-on Wor...
byUiPathCommunity
 
10 Things AI-First Apps Do Differently by iProgrammer Solutions
byiProgrammer Solutions Private Limited
 
Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]
byUiPathCommunity
 
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
Unlocking the Power of Salesforce Architecture: Frameworks for Effective Solu...
byvarsha30tiwari
 
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
Day 4 - Access, Deployments, and Monitoring - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
Our Digital Tribe_ Cultivating Connection and Growth in Our Slack Community 🌿...
bysanjeetmishra30
 
Cross-Cultural Agile Development -Challenges and Strategies for Overcoming Them-
byTakashi Makino
 
communication-skills-with-technology tools
byJaleto Sunkemo
 
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 

Recent advances in deep recommender systems

  • 1.
    Recent Advances inRecommender Systems using Deep Learning August 20, 2019 Sungkyunkwan University (SKKU) Prof. Jongwuk Lee
  • 2.
  • 3.
    What are RecommenderSystems? 3 Items Recommendation products, movies, news, …
  • 4.
    Search vs. Recommendation How can we help users get access to relevant data?  Pull mode (search engines)  Users take initiative.  Ad-hoc information need  Push mode (recommender systems)  Systems take initiative.  Stable information need or a system has user’s information need. 4
  • 5.
  • 6.
    Movie & MusicRecommendation 6 Netflix movie recommendation Spotify music recommendation
  • 7.
  • 8.
    Various Recommendations 8 Social/Friend recommendationRestaurant recommendation Tag recommendationApp recommendation
  • 9.
    What is CollaborativeFiltering?  Given a target user, Alice, find a set of users whose preference patterns are similar to that of the target user.  Predict a list of items that Alice will be likely to prefer. 9 Target user: Alice ① Inferring Alice’s preference ② Finding a set of users with similar preference for Alice ③ Recommending a list of items that a user group prefers Top-N Recommendation
  • 10.
    User-Item Rating Matrix A user-item rating matrix R of the target user, Alice, and other users is given:  R: a user-item rating matrix (𝒎 × 𝒏 matrix)  Determine whether She would like or dislike movies, which has not seen yet. 10 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … … … … … … Alice
  • 11.
    Latent Factor Models How to model user-item interactions?  U: latent user matrix (𝒎 × 𝒌 matrix)  Each user is represented by a latent vector (1 × 𝑘 vector).  V: latent item matrix (𝒏 × 𝒌 matrix)  Each item is represented by a latent vector (1 × 𝑘 vector). 11 User-item interaction 𝒇 𝑼, 𝑽 = ? …… ……
  • 12.
    Factorizing Two LatentMatrices  The user-item rating matrix R can be approximated as a linear combination of two latent matrices U and V.  R: user-item rating matrix (𝑚 × 𝑛 matrix)  U: latent user matrix (𝑚 × 𝑘 matrix)  V: latent item matrix (𝑛 × 𝑘 matrix)  𝑘: # of latent features 12 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … k features kfeatures R U VT … … … … … Yehuda Koren, “Factorization meets the neighborhood: a multifaceted collaborative filtering model,” KDD 2008
  • 13.
    Limitation of ExistingModels  Existing models are mainly based on a linear user-item interaction.  However, the user-item interaction may be non-linear and non-trivial. 13 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5… … k features kfeatures R U VT … … … … …
  • 14.
  • 15.
    Statistics of RecSysModels using DNNs  The number increases exponentially in the last five years.  SIGIR, WWW, RecSys, KDD, AAAI, WSDM, NIPS, … 15 Shuai Zhang et al., “Deep Learning based Recommender System: A Survey and New Perspectives,” 2017
  • 16.
    Categories of RecSysModels using DNNs 16 Deep Learning based Recommender System Model using Single DL Technique Integrate DL with Traditional RS Deep Composite Model Recommend Rely Solely on DL MLP AE CNN RNN DSSM RBM NADE GAN Loosely Coupled Model Tightly Coupled Model Integration ModelNeural Network Model
  • 17.
    Categories of RecSysModels using DNNs 17 Deep Learning based Recommender System Model using Single DL Technique Integrate DL with Traditional RS Deep Composite Model Recommend Rely Solely on DL MLP AE CNN RNN DSSM RBM NADE GAN Loosely Coupled Model Tightly Coupled Model Integration ModelNeural Network Model
  • 18.
    AutoRec: Autoencoder-based Model For each item, reconstruct rating vectors.  Observed ratings are only used to update the model. 18 2 1 3… 2 1 3… …𝒉(𝒓) 𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … Suvash Sedhain et al. “AutoRec: Autoencoders Meet Collaborative Filtering,” WWW 2015
  • 19.
    Denoising Autoencoder (DAE) Learn to reconstruct a user’s favorite set ෤𝒓 of items from randomly sampled subsets, i.e., denoising autoencoder. 19 3 2… 3 3 2… …𝒉(𝒓) ෤𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ Denoising input 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … Yao Wu et al., “Collaborative Denoising Auto-Encoders for Top-N Recommender Systems,” WSDM 2016
  • 20.
    Collaborative Denoising Autoencoder Learn to reconstruct a user’s favorite set ෤𝒓 of items from randomly sampled subsets, i.e., denoising autoencoder.  Train for all users by shared variables for items and a specific variable for each user, i.e., 𝒌 × 𝟏 vector. 20 3 2… 3 3 2… …𝒉(𝒓) ෤𝒓 ො𝒓 𝑾, 𝒃 𝑾′ , 𝒃′ 1 0 0… 𝑽 𝒌×𝟏 One-hot vector for user Denoising input 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … …
  • 21.
    Generalized Matrix Factorization(GMF)  Embeddings are used to learn latent user and item features.  Input: one-hot feature vector for user u and item i  Output: predicted score ŷui  Element-wise product is same as the existing MF model. 21 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Element-wise product Fully connected w/o bias Layer … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … … Xiangnan He et al., “Neural Collaborative Filtering,” WWW 2017
  • 22.
    Step 1: EmbeddingUsers and Items  User embedding  Represent a latent feature for each user.  Item embedding  Similarly, it represents a latent feature for each item. 22 𝟎 𝟏 𝟎 … 𝟎 × 𝟎. 𝟔 𝟎. 𝟖 𝟎. 𝟔 𝟎. 𝟒 𝟎. 𝟐 𝟎. 𝟓 ⋯ 𝟎. 𝟔 𝟏. 𝟐 𝟎. 𝟐 ⋮ ⋱ ⋮ 𝟎. 𝟕 𝟎. 𝟓 ⋯ 𝟎. 𝟑 = 𝟎. 𝟔 𝟎. 𝟒 … 𝟏. 𝟐 𝟏 × 𝒎 vector 𝒎 × 𝒌 matrix 𝟏 × 𝒌 vector 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … …
  • 23.
    Step 1: EmbeddingUsers and Items 23 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … R U VT … … … … … …
  • 24.
    Step 2: Element-wiseProduct  This is same as the linear combination of U and VT. 24 Element-wise product Layer 𝟎. 𝟕𝟐 … 𝟎. 𝟑𝟔 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … … … … … … 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 R U VT
  • 25.
    Step 3: PassingFully Connected Layer 25 ෝ𝒚 𝒖𝒊 Fully connected w/o bias 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … 1.5 ... 0.1 0.6 ... 1.2 0.7 ... 0.5 … ... ... 0.1 ... 0.2 0.2 1.4 1.2 … 2.3 … ... … ... ... 0.1 2.6 0.3 … 1.5 … … R … … … … … If the weight is equal, it is same as the existing MF model. 𝑹 = 𝒘(𝑼⨂𝑽 𝑻) Element-wise product Layer 𝟎. 𝟕𝟐 … 𝟎. 𝟑𝟔 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector User embedding Item embedding … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 U VT
  • 26.
    MLP-based Matrix Factorization Instead of element-wise product, concatenate latent user and item vectors. 26 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Concatenating two latent vectorsLayer 1 Layer X ……. Fully connected w/o bias … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … …
  • 27.
    MLP-based Matrix Factorization Learning non-trivial interactions between users and items 27 0 0 1 … 00 1 0 … 0 latent user vector Latent item vector ෝ𝒚 𝒖𝒊 User embedding Item embedding Concatenating two latent vectorsLayer 1 Layer X ……. Fully connected w/o bias … … 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … R … … … … … 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑 𝟎. 𝟔 … 𝟏. 𝟐 𝟏. 𝟐 … 𝟎. 𝟑
  • 28.
    Fusing Two Solutions Neural collaborative filtering: Both use GMF and MLP layers. 28 0 0 1 … 00 1 0 … 0 MF user vector MLP item vectorMLP user vector MF item vector MLP Layer 1 MLP Layer X ……. Element-wise product GMF Layer ෝ𝒚 𝒖𝒊 Concatenation Fully connected w/o bias NeuMF Layer Concatenation ……
  • 29.
  • 30.
    Distributional Hypothesis  “Youshall know a word by the company it keeps.” by J.R. Firth (1957)  Words with similar contexts share similar meanings.  One of the most successful ideas of modern NLP  What is Tejuino ?  A cup of Tejuino is on the table.  A woman likes Tejuino.  Tejuino makes you drunk.  I usually drink Tejuino every morning. 30
  • 31.
    Prod2Vec using WordEmbeddings  How to perform product embedding?  Adopt the skip-gram model for products.  Input: i-th product purchased by the user  Output: the other product purchased by the user 31 A set of words  A set of products purchased by the user Mihajlo Grbovic et al., “E-commerce in Your Inbox: Product Recommendations at Scale,” KDD 2015
  • 32.
    Prod2Vec using WordEmbeddings  Imagine that the existing user-item matrix.  Word  Movie, A set of words  User  The window size is ignored. 32 3 3 ? 2 ? ? 4 1 5 4 ? ? 3 ? ? 3 … … … … … … i-th movie watched by the user Projection Softmax Transform The other movie watched by the user
  • 33.
    Possible Models ofProd2Vec 33 i-th product purchased by the user Projection Softmax Transform The other product purchased by the user i-th product purchased by the user Projection + Averaging Softmax Transform user products except for the i-th product purchased by the user Prod2Vec Skip-gram model User embedding + Prod2Vec
  • 34.
    34 About Time (2013) 드라마/판타지/성장 어떠한순간을 다시 살게 된다면, 과연 완벽한 사랑을 이룰 수 있을까? 모태솔로 팀은 성인이 된 날, 아비저로부터 놀랄만한 가문의 비밀을 듣게 된다. 바로 사긴을 되돌릴 수 있는 능력이 있다는 것! 여자친구를 만든다는 꿈을 이루기 위해 런던으로 간 팀은 메리에게 첫눈에 반하게 되는데.. Silver Linings Playbook (2012) 드라마/코미디 Secret Life of Walter Mitty, The (2013) 판타지/드라마 Perks of Being a Wallflower, The (2012) 드라마/성장 The Theory of Everything(2014) 드라마/로맨스 What If (2013) 로맨스/코미디 Man Up (2015) 드라마/로맨스 Love, Rosie (2014) 로맨스/코미디 Two Night Stand (2014) 로맨스/코미디 GloVe Skip-gram
  • 35.
    Mulan (1998) 애니메이션 동양의 분위기를살린 디즈니의 역작! 파씨 가문의 외동딸 뮬란은 선머슴 같은 성격 때문에 중매를 볼 때마다 퇴짜를 맞는다. 때마침 훈족의 침입으로 징집명령이 떨어지고 늙은 아버지를 대신하여 남장을 하고 나서는데.. Jungle Book, The (1967) 애니메이션 Antz (1998) 애니메이션 Lady and the Tramp (1955) 애니메이션 Peter Pan (1953) 애니메이션 Thumbelina (1994) 애니메이션 A Dinosaur's Story (1993) 애니메이션 Quest for Camelot (1998) 애니메이션 Return to Never Land (2002) 애니메이션 35 GloVe Skip-gram
  • 36.
    Machine, The (2013) 판타지/SF(로봇) 인간과로봇 그 경계가 사라진다! Signal, The (2014) 스릴러/SF (컴퓨터) Zero Theorem, The (2013) 판타지/드라마(컴퓨터) Autómata (2014) 스릴러/액션(로봇) Cargo (2009) 스릴러/미스터리(우주) 신 냉전시대에 인간의 뇌 데이터를 바탕으로 탄생한 살인로봇 머신 에이바는 점차 인간의 감정을 느껴가고, 그녀를 주축으로 머신들은 인간과의 최후의 전쟁을 선포하는데… the east(2013) 스릴러/액션(스파이) Signal, The (2014) 스릴러/SF (컴퓨터) Autómata (2014) 스릴러/액션(로봇) Cargo (2009) 스릴러/미스터리(우주) 36 GloVe Skip-gram
  • 37.
    Session-based Recommendation  Sequentialbehavior (global information)  Last click (local information) 37 Session input Recommendation
  • 38.
    GRU4Rec: RNN-based Model 38 Item2Vec RNNLayer Balazs Hidasi et al., “Session-based Recommendations with Recurrent Neural Networks”, RecSys Workshop 2015
  • 39.
    NARM: Attention-based Model Recommend the next item in a given session.  Combine global and local information.  They are represented by RNNs. 39 Jing Li et al., “Neural Attentive Session-based Recommendation,” CIKM 2017
  • 40.
    Step 1: GlobalEncoder in NARM  Capturing user’s sequential behavior  ℎ 𝑡 = 1 − 𝑧𝑡 ℎ 𝑡−1 + 𝑧𝑡 ෠ℎ 𝑡  𝑧𝑡 = 𝜎(𝑊𝑧 𝑥 𝑡 + 𝑈𝑧ℎ 𝑡−1) where 𝑧𝑡 is update gate  ෠ℎ 𝑡 = tanh[𝑊𝑥 𝑡 + 𝑈 𝑟𝑡 ⊙ ℎ 𝑡−1 ]  𝑟𝑡 = 𝜎(𝑊𝑟 𝑥 𝑡 + 𝑈𝑟ℎ 𝑡−1) 40
  • 41.
    Step 2: LocalEncoder in NARM  Capturing user’s main purpose  𝛼 𝑡𝑗 = 𝑞 𝒉 𝒕, ℎ𝑗 , where 𝒉 𝒕 is a latent vector for the last item.  𝐶𝑡 𝑙 = σ 𝑗=1 𝑡 𝛼 𝑡𝑗ℎ𝑗  𝑞 is an attention scoring function for ℎ 𝑡 and ℎ𝑗.  𝑞 ℎ 𝑡, ℎ𝑗 = 𝑣 𝑇 𝜎(𝐴1ℎ 𝑡 + 𝐴2ℎ𝑗) 41
  • 42.
    Step 3: Decoderin NARM  Concatenated vector 𝑐𝑡 = 𝑐𝑡 𝑔 ; 𝑐𝑡 𝑙 = [ℎ 𝑡 𝑔 ; σ 𝑗−1 𝑡 𝛼 𝑡𝑗ℎ 𝑡 𝑙 ]  Use an alternative bi-linear similarity function.  𝑆𝑖 = 𝑒𝑚𝑏𝑖 𝑇 𝐵𝑐𝑡 where 𝐵 is a 𝐷 × |𝐻| matrix.  |𝐷| is the dimension of each item. 42
  • 43.
    Combining Attention andMemory  𝒎 𝒔 represents the average vector of items.  𝑚 𝑠 = 1 𝑡 σ𝑖=1 𝑡 𝑥𝑖  𝒎 𝒕 is the vector for last item. 43 Qiao Liu et al., “Short-Term Attention/Memory Priority Model for Session-based Recommendation,” KDD 2018
  • 44.
    STAMP: Attention/Memory Priority 𝒎 𝒂 is sum of multiplication between coefficient and embedding vector.  𝑚 𝑎 = σ𝑖=1 𝑡 𝛼𝑖 𝑥𝑖  Attention coefficient: 𝛼𝑖 = 𝑊0 𝜎(𝑊1 𝑥𝑖 + 𝑊2 𝒎 𝒔 + 𝑊3 𝒙 𝒕 + 𝑏) 44
  • 45.
    MMCF: Multimodal CollaborativeFiltering for Automatic Playlist Continuation RecSys Challenge 2018 Team ‘hello world!’ (2nd place), main track Hojin Yang, Yoonki Jeong, Minjin Choi, and Jongwuk Lee Sungkyunkwan University, Republic of Korea
  • 46.
    Automatic Playlist Continuation Million Playlist Dataset (MPD) 46 Playlist title Tracks in the playlist Metadata of tracks (artist, album)
  • 47.
    Challenge Set 47 1 23 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Few tracks in the first part Many tracks in the first part Many tracks in the random position
  • 48.
    Challenge Set 48 1 23 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 No tracks in the playlist How to deal with an edge case?
  • 49.
    Challenge Set 49 1 23 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Scarce information How to treat playlists with scarce information?
  • 50.
    Challenge Set 50 1 23 4 5 6 7 8 9 10 # of tracks 0 1 5 10 5 10 25 100 25 100 Title available Yes Yes Yes Yes No No Yes Yes Yes Yes Track order Seq Seq Seq Seq Seq Seq Seq Seq Shuffled Shuffled # of playlists 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000 Various types of Input How to deal with various types of input?
  • 51.
    Overview of theProposed Model  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 51
  • 52.
    Overview of theProposed Model  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 52
  • 53.
    Autoencoder-based Model 53 1 0 1 1 0 1Hey Jude Rehab Yesterday DancingQueen Mamma Mia Viva la Vida encoder decoder  Learn a latent representation of a given playlist consisting of a set of tracks 0.9 0.01 0.78 0.9 0. 6 0.8 ✔ Top-1 Recommendation Output
  • 54.
    Denoising Autoencoder 54 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8HeyJude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder 1 0 1 1 0 1  Training with denoising  Some positive input values are corrupted (set to zero). denoising
  • 55.
    Denoising Autoencoder 55  Trainingwith denoising  Some positive input values are corrupted (set to zero). How to utilize the metadata such as artists and albums? 1 0 1 1 0 1 1 0 1 0 0 0 0.9 0.01 0.78 0.9 0. 6 0.8Hey Jude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder 1 0 1 1 0 1 denoising
  • 56.
    Utilizing Metadata 56 1 0 1 1 0 1 0 1 1 0 0.9 0.01 0.78 0.9 0. 6 0.8 0.2 0.98 0.9 0.6 HeyJude Rehab Yesterday Dancing Queen Mamma Mia Viva la Vida encoder decoder  Concatenate an artist vector corresponding to the track vector in the playlist.
  • 57.
     Randomly chooseeither the playlist or its artists as input. 57 Training Strategy: Hide-and-Seek
  • 58.
    Training Strategy: Hide-and-Seek Randomly choose either the playlist or its artists as input. 58
  • 59.
    Training Strategy: Hide-and-Seek Randomly choose either the playlist or its artists as input. 59
  • 60.
    CharCNN for PlaylistTitles  An ensemble method with two components.  Autoencoder for tracks and metadata for tracks.  CharCNN for playlist titles. 60
  • 61.
    Word-level CNN forNLP  Effective for capturing spatial locality of a sequence of texts 61 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filter (3 by k ) 2.2 2.3 -1.3 0.9 max pooling Conv layer 2.3 Feature much convolution k-dimension embedding
  • 62.
    Word-level CNN forNLP  Effective for capturing spatial locality of a sequence of texts 62 I like this song very much 0.1 0.3 0.2 0.6 0.2 0.6 -1.2 -0.2 -2.1 0.2 0.1 0.4 -2.1 0.9 -3.1 1.4 0.1 0.3 -0.2 0.1 0.4 0.1 0.7 0.1 I like this song very Filters (3 by k ) convolution 2.2 2.3 -1.3 0.9 max pooling 2.3 Feature much k-dimension embedding Conv layer 2.2 2.3 -1.3 0.9 Conv layers 2.2 2.3 -1.3 0.9 1.2 2.4 -1.1 0.4 max pooling 2.3 1.2 2.4 Feature vector convolution
  • 63.
    CharCNN for PlaylistTitles  Playlist titles are represented by a short text, implying an abstract description of a playlist.  Use character-level embedding. 63 Conv layers Feature vector
  • 64.
    Combining Two Models Simplest method: 𝑤𝒊𝒕𝒆𝒎 = 0.5 and 𝑤𝒕𝒊𝒕𝒍𝒆 = 0.5 64
  • 65.
    Combining Two Models The accuracy of the AE highly relies on the number of tracks within a playlist.  Dynamic: Set weights according to the number of items. 65 Items Playlist Title Chill songs 0.7 0.4 0.9 0.1 0.2 0.1 0.2 0.3 0.7 0.1 0.6 0.4 0.7 0.2 0.2 AE CNN 𝑤_𝑖𝑡𝑒𝑚 = 5 𝑤_𝑡𝑖𝑡𝑙𝑒 = 1
  • 66.
  • 67.
    Recent Progress inOur Lab  “Dual Neural Personalized Ranking,” WWW 2019  “Characterization and Early Detection of Evergreen News Articles,” ECML/PKDD 2019 (To appear)  “Collaborative Distillation for Top-N Recommendation,” ICDM 2019 (To appear) 67
  • 68.
  • 69.
    SR-GNN using GraphNeural Nets  Each session graph is proceeded one by one. Node vectors can be obtained through a gated graph neural network.  Each session is represented as the combination of the global preference and current interests of this session using an attention net. 69 Shu Wu et al., “Session-based Recommendation with Graph Neural Networks,” AAAI 2019
  • 70.