Gabriel Moreira, profile picture
Uploaded byGabriel Moreira
PDF, PPTX1,355 views

[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

The document presents a doctoral thesis defense by Gabriel de Souza Pereira Moreira focused on 'Chameleon', a deep learning meta-architecture for improving session-based news recommendation systems. It addresses challenges like preference shifts and item cold-start issues, and outlines objectives such as enhancing recommendation quality through a recurrent neural network-based architecture. The research emphasizes personalized recommendations leveraging contextual user information and aims to balance factors of accuracy, novelty, and diversity in recommendations.

Technology◦
Related topics:
Deep Learning•

Embed presentation

Download as PDF, PPTX
CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019
Introduction "We are leaving the Information Age and entering the Recommendation Age.". Cris Anderson, "The long tail" 2
News consumption is the majority of web traffic (TREVISIOL et al. , 2014b) 3 Introduction
4 Research trends on News Recommender Systems (KARIMI et al. , 2018) Introduction
5 Research trends on News Recommender Systems (KHAN et al., 2019) Introduction
6 1. Preferences shift Introduction News Recommender Systems Challenges
7 1. Preferences shift 2. Sparse user profiles Introduction News Recommender Systems Challenges # sessions by user (G1 dataset, 16 days) Q3 = 3 sessions Q2 = 2 sessions Q1 = 1 session
8 1. Preferences shift 2. Sparse user profiles 3. Fast growing number of items Introduction News Recommender Systems Challenges # fresh articles by day (G1 dataset)
9 1. Preferences shift 2. Sparse user profiles 3. Fast growing number of items 4. Accelerated item’s value decay Introduction News Recommender Systems Challenges Permanent User and Item Cold-Start Problem Articles age at click time distribution (G1 dataset) Percentile of # clicks Article Age 10% < 4 hours 25% < 5 hours 50% < 8 hours 75% < 14 hours 90% < 26 hours
Objectives 10 1. To investigate the quality of news recommendation using session-based algorithms 2. To design a Deep Learning meta-architecture for hybrid and contextual recommendation, based on Recurrent Neural Networks, to improve the quality of recommendations provided by news portals.
Research Questions ● How does the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 11 ● What is the effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? ● What is the effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? ● Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? ● Is the proposed approach effective to balance the competing quality factors of accuracy and novelty? 1 2 3 4 5
Scope (1/2) ● Session-based news recommendation ● Task: Next-click prediction for user sessions in a news portal 12
Scope (1/3) ● Session-based news recommendation ● Task: Next-click prediction for user sessions in a news portal 13 User session clicks C1 C2 C3 C4 Next-click prediction model Article B Article A Article C Article D ... Ranked articles Recommendable articles (sampled)
Scope (2/3) ● The proposed Deep Learning meta-architecture should be: ○ Session-based ○ Hybrid ○ Contextual 14 ● Recommendation quality factors to be evaluated: ○ Accuracy ○ Item Coverage ○ Novelty ○ Diversity ○ Robustness against the Item Cold-Start Problem
Recommender System Content-based filtering Collaborative filtering Model-based filteringMemory-based filtering Item-basedUser-based ML-based: Clustering, Association Rules, Matrix Factorization, Neural Networks Hybrid filtering+ = Focus of this research 15 Scope (3/3)
Deep Learning for Recommender Systems
Deep Learning adoption in RecSys 2007 2015 2016 2017-2019 Deep Boltzmann Machines for rating prediction calm before the storm A few seminal papers First DLRS workshop and papers on RecSys, KDD, SIGIR Continued increase 17
Research directions in DL-RecSys And their combinations... Session-based recommendations with RNNs Feature Extraction directly from the content Learning Item embeddings Deep Collaborative Filtering Multi-view Learning Focus of this research 18
Related work 19 Main inspirations: ● GRU4Rec (HIDASI, 2016) - The seminal work on the usage of Recurrent Neural Networks (RNN) on session-based recommendations, and subsequent work (Hidasi,2017). ● MV-DNN (ELKAHKI, 2015) - Adapted Deep Structured Semantic Model (DSSM) for the recommendation task. MV-DNN maps users and items to a latent space, where the cosine similarity between users and their preferred items is maximized. That approach makes it possible to keep the neural network architecture static, rather than adding new units into the output layer for each new item.
CHAMELEON: A Deep Learning Meta-Architecture for News Recommendation
The conceptual model of news relevance factors 2121 News relevance Topics Entities Publisher News static properties Recency Popularity News dynamic properties News article User TimeLocation Device User current context Long-term interests Short-term interests Global factors Season- ality User interests Breaking events Popular Topics Referrer
Meta-Architecture Requirements (1/2) ● RR1 - to provide personalized news recommendations in extreme cold-start scenarios, as most news are fresh and most users cannot be identified ● RR2 - to automatically learn news representations from textual content and news metadata, minimizing the need of manual feature engineering ● RR3 - to leverage the user session information, as the sequence of interacted news may indicate user's short-term preferences for session-based recommendations ● RR4 - to leverage users' contextual information, as a rich data source in such information scarcity about the user 22
Meta-Architecture Requirements (2/2) ● RR5 - to model explicitly contextual news properties (popularity and recency), as those are important factors on news interest life cycle ● RR6 - to support an increasing number of new items and users by incremental model retraining (online learning), without the need to retrain on the whole historical dataset ● RR7 - to provide a modular structure for news recommendation, allowing its modules to be instantiated by different and increasingly advanced neural network architectures and methods 23
The CHAMELEON A master on context adaptation 24
The CHAMELEON Each eye can pivot independently, allowing to focus two different objects simultaneously. 25
The CHAMELEON An accurate hunter 26
The CHAMELEON Meta-Architecture for News RS 27 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Representation (ACR) module Responsible for learning distributed representations (embeddings) from news’ contents. Sub-modules: ● Textual Features Representation (TFR) ● Content Embeddings Training (CET) 28 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 29 Inputs Article text represented by Word embeddings Word embeddings pre-trained in larger corpus: ● Word2Vec ● GloVe Article Content Embeddings When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Article Content Embedding (ACE)
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Textual Features Representation (TFR) sub-module ● Learns relevant features from article textual content (RR2). ● Can be instantiated by a Deep NLP architecture like: ○ CNN ○ RNN ○ QRNN ○ Attention mechanisms 30 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Embedding (ACE) Content Embeddings Training (CET)
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Embedding ● The learned article distributed representation ● After training, it is stored in a repository for further usage by NAR module 31 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Content Embeddings Training (CET)
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Content Embeddings Training (CET) ● Responsible to train Article Content Embeddings. ○ Supervised task: e.g. predict article metadata attributes ○ Unsupervised task: e.g. reconstruct a compressed version of the input (autoencoder) ● May be instantiated as a sequence of feed-forward Fully Connected layers. 32 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Next-Article Recommendation (NAR) module ● Provide news articles recommendations for each interaction (I) in active user sessions. ● For each recommendation request, NAR module generates a ranked list of the most likely articles user might read in a given session. Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) Time Location Device User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session 33
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 1 - Article Content Embedding Pre-trained by the ACR module (RR2) 2 - User Context Properties of user current context (RR4): ○ Time (Period of day, Weekday / Weekend, Month, Quarter) ○ Location (City, State, Country) ○ Device (eg. Desktop / Smartphone) ○ Device OS (eg. Android / iOS) 34
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 3 - Article context Contextual properties of the article (RR5): ● Recent Popularity - Popularity of the article within the last hour ● Recency - Time elapsed since the article was published 35
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 36 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Contextual Article Representation (CAR) sub-module ● Merges the inputs – article content, article context and user context – and generates the User-Personalized Contextual Article Embedding ● This sub-module might be instantiated as Fully Connected (FC) layers or using Factorization Machines (FM), for example.
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 37 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sessions in a batch I1,1 I1,2 I1,3 I1,4 I1,5 I2,1 I2,2 I3,1 I3,2 I3,3
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 38 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sampling strategy Negative samples: Articles read by other users in the last hour Positive sample: Next article read by the user in his session Samples
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: SEssion Representation (SER) sub-module ● Models short-term user’s preferences in their active sessions (RR3) ● Must be instantiated by an architecture that is able to model the sequence of user clicks, such as RNN, CNN, attention mechanisms. 39 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER)
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Predicted Next-Article Embedding ● Represents the expected representation of a news content the user would like to read next in the active session. 40 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module ● Recommends articles to a user session based on embeddings similarity, supporting unseen items during inference (RR6). ● The loss function maximizes the similarity between “Predicted Next-Article Embedding” and the “User-Personalized Contextual Article Embedding” corresponding to the positive sample, and minimize similarities with negative samples. 41 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommendations Ranking (RR)
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 42 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 1 - Relevance Score of an item for a user session Eq. 2 - Matching function Eq. 3 - Softmax over Relevance Score Eq. 4 - Accuracy Loss function
The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 43 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 5 - Novelty Loss function Eq. 6 - Novelty Eq. 7 - Recent Normalized Popularity Eq. 8 - Multi-objective Loss Function
The CHAMELEON Meta-Architecture for News RS 44 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings
45 Comparison among Deep Architectures for News RS Characteristics (SONG, 2016) (KUMAR, 2017) (PARK, 2017) (OKURA, 2017) (WANG, 2018) (ZHANG, 2018) CHAMELEON Session-based Multiple quality objectives Temporal articles relevance decay (exponential function) (decay learned by the model) Time-aware negative sampling N/A - Does not use negative sampling Article Context (Recency, Recent Pop) User Context Textual Content Features Letter trigrams, processed by MLP doc2vec PV-DBoW Denoising auto-encoder Entities enrichment of title with a Knowledge Graph Character-level CNN Sup. and Unsup. ACR instantiations (CNN, GRU) DNN architecture for sequence modeling RNN RNN RNN RNN DNN with attention RNN RNN Loss function Listwise ranking based on DSSM Listwise ranking based on DSSM Pairwise ranking BRP, TOP1 Pairwise ranking with position bias Pointwise ranking: Log loss Pairwise ranking: BRP, TOP1 Listwise ranking based on DSSM (custom)
CHAMELEON Architecture Instatiations
ACR module - Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Output: Article Metadata Attributes ● If more than one attribute is available for classification, Multi-Task Learning (MTL) is used. TASK A TASK B TASK C Task-specific layers Shared layers 47 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers When a news article is published...
ACR module - Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 48 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers Output: Article Metadata Attributes ● The last layer of ACR module is responsible to classify one or more article metadata attributes ● Each attribute might be single-label (e.g. category) or multi-label (e.g. tags), in Eq. 9 and 10, respectively Eq. 9 - Softmax Eq. 10 - Sigmoid Eq. 11 - Cross-entropy When a news article is published...
ACR module - Supervised - RNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 49 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers GRU encoderinput word embeddings hidden state RNN based of TFR sub-module ● Process the sequence of word embedding using an RNN ● P.s. Not as computationally efficient as CNNs
ACR module - Unsupervised Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 50 Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) GRU encoder Fully Connected layer Reconstructed word embeddings Fully Connected layer input word embeddings hidden state GRU decoder New York is a multicultural city , ... hidden state Article Content Embeddings Word Embeddings Autoencoder based instantiation of the ACR module ● A Sequence Denoising Autoencoder ● The objective is to reconstruct the sequence of corrupted input word embeddings from a compressed representation (Article Content Embedding) ● P.s. The sequence of input words is reversed to improve reconstruction accuracy
51 NAR module instantiation Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active articleActive Sessions Predicted Next-Article Embedding Session Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Fully Connected layers UGRNN + Fully Connected layers Time Location Device Referrer User interaction When a user reads a news article... Active User Session NAR module instantiation ● Learnable parameters for centering and scale input features ● CAR sub-module was instantiated by 2 FC layers (Leaky ReLU + tanh) ● SER sub-module was instantiated by a 2-layer UGRNN cell (COLLINS, 2017) followed by 2 FC layers (Leaky ReLU + tanh).
CHAMELEON Implementation 52 ● CHAMELEON’s instantiations are implemented using TensorFlow https://github.com/gabrielspmoreira/chameleon_recsys ● Training and evaluation performed in Google Cloud Platform ML Engine
Experiments Design
Evaluation Protocol 54 Task: For each item within a session, predict the next-clicked item from a set composed by the positive sample (correct article) and 50 negative samples. Hours Input I1 I2 I3 I4 I2 I3 I4 I5 Expected Output (labels)
Metrics (1/2) 55 Accuracy HR@10 Hit Rate - Checks whether the positive item is among the top-N ranked items MRR@10 Mean Reciprocal Rank - Ranking metric which assigns higher scores at top ranks Item Coverage COV@10 # recommended articles @ top-n # recommendable articles
Metrics (2/2) Novelty ESI-R@10 ESI-RR@10 56
Metrics (2/2) 57 Novelty ESI-R@10 ESI-RR@10 Diversity EILD-R@10 EILD-RR@10
Datasets (1/2) 58 G1 dataset ● Provided by Globo.com, the most popular news portal in Brazil ● Made publicly available by this research for the community https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom
59 G1 (globo.com) Adressa Language Portuguese Norwegian Period (days) 16 16 # users 322 k 314 k # sessions 1,048 k 982 k # clicks 2,988 k 2,648 k # articles 46 k 13 k Avg. session length * 2.84 2.70 * After removing single-click sessions and sessions with more than 20 clicks. Datasets (2/2)
Baseline algorithms 60 Neural methods GRU4Rec A landmark neural architecture using RNNs for session-based recommendation SR-GNN A recently state-of-the-art architecture for session-based recommendation based on GNN. Frequent patterns methods CO A simple method of co-occurrence based on association rules of length two. SR Sequential rules of size two. kNN methods Item-kNN Most similar articles to the last read one, in terms of the cosine similarity between the vector of their occurrence in sessions. V-SkNN Compares the entire active session with past (neighboring) sessions to determine items to be recommended. The similarity function emphasizes items that appear later within the session. Non-personalized methods RP Recently Popular - Most viewed articles during the last hour. CB Content-Based - Most similar articles (content embedding) to the last user click.
Main Results
Research Questions How does the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality factors (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 62 DLRS @ RecSys, 2018 IEEE Access, 2019 (Qualis A2, Eng. IV) 1
G1 dataset +14.2% +19.6% News recommendation quality of session-based algorithms 1 63 Other neural-based methods
+14.2% +19.6% Adressa dataset +11.6% +13.2% News recommendation quality of session-based algorithms 1
65 Average MRR@10 by hour (evaluation each 5 hours), during 16 days G1 dataset News recommendation quality of session-based algorithms 1
66 Average MRR@10 by hour (evaluation each 5 hours), during 16 days Adressa dataset News recommendation quality of session-based algorithms 1
67 Recommendation Accuracy (HR@10 ) of algorithms (lines) x Avg. Normalized Popularity (bars) by session click order Adressa dataset News recommendation quality of session-based algorithms 1 G1 dataset
News recommendation quality of session-based algorithms 68 Evaluation of other quality factors G1 dataset 1
News recommendation quality of session-based algorithms 69 Evaluation of other quality factors 1Adressa dataset
Research Questions 70 What is the effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? IEEE Access, 2019 (Qualis A2, Eng. IV) 2
71 The effect of different types of information 2 Input Configuration Feature Sets IC1 Article Id IC2 IC1 + Article Context (Novelty and Recency) IC3 IC2 + the Article Content Embeddings (ACE) learned by the supervised instantiation of the ACR module IC4 IC3 + Article Metadata IC5 IC4 + User Context
72 The effect of different types of information 2
Research Questions 73 What is the effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? INRA @ RecSys, 2019 3
74 The effect of different content representations 3ACRmodule instantiations
75 Recommender HR@10 MRR@10 No-ACE 0.6281 0.3066 Supervised CNN 0.6585 0.3395 GRU 0.6585 0.3388 Unsupervised SDA-GRU 0.6418 0.3160 W2V*TF-IDF 0.6575 0.3291 LSA 0.6686* 0.3423 doc2vec 0.6368 0.3119 Comparison of different content encodings for CHAMELEON The effect of different content representations G1 dataset 3
76Comparison of different content encodings for CHAMELEON The effect of different content representations Adressa dataset 3 Recommender HR@10 MRR@10 No-ACE 0.6816 0.3252 Supervised CNN 0.6860 0.3333 GRU 0.6856 0.3327 Unsupervised SDA-GRU 0.6905 0.3360 W2V*TF-IDF 0.6913 0.3402 LSA 0.6935 0.3403 doc2vec 0.6898 0.3402
Research Questions Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? 77 4
78 The effect on item cold-start 4 # Batches before First Recommendation (BFR@n) G1 dataset
79 The effect on item cold-start 4 Adressa dataset # Batches before First Recommendation (BFR@n)
Research Questions Is the proposed approach effective to balance the competing quality factors of accuracy and novelty? 80 IEEE Access, 2019 (Qualis A2, Eng. IV) 5
81 Balancing the trade-off between Accuracy and Novelty 5 G1 dataset Adressa dataset
Conclusion
Main Contributions 83 1. The investigation of the stated research questions related to the effectiveness of session-based recommendation algorithms on the news domain 2. The design, implementation, and evaluation of a Deep Learning meta-architecture – the CHAMELEON – for hybrid and contextual session-based news recommendation
Complementary Contributions 84 1. The elaboration of a Conceptual Model of factors that affect news relevance 2. The four instantiations of ACR and NAR modules of CHAMELEON 3. The open-source implementation of CHAMELEON 1 4. The temporal offline evaluation protocol for news recommendation 5. The adaptation of rank- and relevance-sensitive novelty and diversity metrics Additional Contribution 6. The preprocessing and sharing of two novel datasets for evaluation of hybrid and contextual RS for news (G1) 2 and articles (CI&T Desktop) 3 1 https://github.com/gabrielspmoreira/chameleon_recsys 2 https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom 3 https://www.kaggle.com/gspmoreira/articles-sharing-reading-from-cit-deskdrop
Recommendations 85 1. Online evaluation of CHAMELEON in a live news portal 2. To balance more than two quality factors (including diversity) 3. To adapt CHAMELEON for session-aware news recommendation 4. To explore different negative sample approaches (e.g., by geographic region) 5. Explore other Deep NLP techniques (e.g. BERT) to produce better article content embeddings 6. To deepen the investigation of the RQ4 about the item cold-start problem (next paper)
Suggestions for Future Works 86 1. Instantiate the SER sub-module with a neural network architecture other than RNNs (e.g., CNN, QRNN, GNN) 2. Investigate attention mechanisms as a way to improve accuracy and provide recommendation explanations 3. Investigate “noisy outliers” in the user profiles 4. Adapt CHAMELEON for contextual and hybrid session-based recommendations in other domains like e-commerce, entertainment and media
Published Articles 87 1. MOREIRA, Gabriel de Souza Pereira; SOUZA JUNIOR, G. J. A. ; CUNHA, A. M. . Comparing offline and online recommender system evaluations on long-tail distributions. In: ACM Recommender Systems Conference, 2015, Viena, Austria. Poster Proceedings of ACM RecSys 2015, 2015. v. 1441. p. 2. MOREIRA, GABRIEL DE SOUZA P.; SOUZA, GILMAR . A Recommender System to tackle Enterprise Collaboration. In: the 10th ACM Conference, 2016, Boston. Proceedings of the 10th ACM Conference on Recommender Systems - RecSys '16 (Demo paper). New York: ACM Press, 2016. p. 395. DOI https://doi.org/10.1145/2959100.2959115 3. MOREIRA, GABRIEL. CHAMELEON: a Deep Learning Meta-Architecture for News Recommender Systems. In: 12th ACM Conference on Recommender Systems - RecSys'18, 2018, Vancouver. Proceedings of the 12th ACM Conference on Recommender Systems RecSys'18 - Doctoral Symposium. New York: ACM Press, 2018. DOI https://doi.org/10.1145/3240323.3240331 4. MOREIRA, GABRIEL; FERREIRA, FELIPE ; DA CUNHA, ADILSON MARQUES . News Session-Based Recommendations using Deep Neural Networks. Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems - DLRS 2018. New York: ACM Press, 2018. p.15. DOI https://doi.org/10.1145/3270323.3270328 5. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES . On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems. Proceedings of the 7th Workshop on News Recommendation and Analytics (INRA 2019), in conjunction with RecSys 2019, September 19, 2019, Copenhagen 6. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES .. "Contextual Hybrid Session-Based News Recommendation With Recurrent Neural Networks." IEEE Access 7 (2019): 169185-169203.
CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019

More Related Content

PDF
Hands on Explainable Recommender Systems with Knowledge Graphs @ RecSys22
byGiacomoBalloccu
 
PDF
Knowledge Graph Embeddings for Recommender Systems
byEnrico Palumbo
 
PDF
Phd thesis final presentation
byCristhian Figueroa
 
PDF
Recommendation engines
byGeorgian Micsa
 
PDF
Deep Learning for Recommender Systems RecSys2017 Tutorial
byAlexandros Karatzoglou
 
PPTX
Recommendation system
byRishabh Mehta
 
PPTX
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
 
PPTX
Recommendation system
byAkshat Thakar
 
Hands on Explainable Recommender Systems with Knowledge Graphs @ RecSys22
byGiacomoBalloccu
 
Knowledge Graph Embeddings for Recommender Systems
byEnrico Palumbo
 
Phd thesis final presentation
byCristhian Figueroa
 
Recommendation engines
byGeorgian Micsa
 
Deep Learning for Recommender Systems RecSys2017 Tutorial
byAlexandros Karatzoglou
 
Recommendation system
byRishabh Mehta
 
Recommender systems: Content-based and collaborative filtering
byViet-Trung TRAN
 
Recommendation system
byAkshat Thakar
 

What's hot

PDF
Introduction to Recommendation Systems
byTrieu Nguyen
 
PPTX
Recommender system
byNilotpal Pramanik
 
PPTX
Recommendation system
byPratisthaSingh5
 
PDF
Recommender Systems
byFrancesco Casalegno
 
PPTX
Recommendation System
byAnamta Sayyed
 
PDF
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
byXavier Amatriain
 
PPTX
Collaborative Filtering using KNN
byŞeyda Hatipoğlu
 
PDF
Sparse Communication for Federated Learning
byKundjanasith Thonglek
 
PPT
Amazon Item-to-Item Recommendations
byRoger Chen
 
PDF
Kdd 2014 Tutorial - the recommender problem revisited
byXavier Amatriain
 
PDF
Incorporating Diversity in a Learning to Rank Recommender System
byJacek Wasilewski
 
PDF
Deep Learning for Recommender Systems
byJustin Basilico
 
PDF
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
byAlexandros Karatzoglou
 
PDF
Time, Context and Causality in Recommender Systems
byYves Raimond
 
PDF
Warsaw Data Science - Factorization Machines Introduction
byBartlomiej Twardowski
 
PDF
Information Retrieval Models for Recommender Systems - PhD slides
byDaniel Valcarce
 
PDF
Recommender system algorithm and architecture
byLiang Xiang
 
PDF
Deep Q-Learning
byNikolay Pavlov
 
PPTX
Deep Learning in Recommender Systems - RecSys Summer School 2017
byBalázs Hidasi
 
PPT
Data Mining: Concepts and techniques classification _chapter 9 :advanced methods
bySalah Amean
 
Introduction to Recommendation Systems
byTrieu Nguyen
 
Recommender system
byNilotpal Pramanik
 
Recommendation system
byPratisthaSingh5
 
Recommender Systems
byFrancesco Casalegno
 
Recommendation System
byAnamta Sayyed
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
byXavier Amatriain
 
Collaborative Filtering using KNN
byŞeyda Hatipoğlu
 
Sparse Communication for Federated Learning
byKundjanasith Thonglek
 
Amazon Item-to-Item Recommendations
byRoger Chen
 
Kdd 2014 Tutorial - the recommender problem revisited
byXavier Amatriain
 
Incorporating Diversity in a Learning to Rank Recommender System
byJacek Wasilewski
 
Deep Learning for Recommender Systems
byJustin Basilico
 
Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial
byAlexandros Karatzoglou
 
Time, Context and Causality in Recommender Systems
byYves Raimond
 
Warsaw Data Science - Factorization Machines Introduction
byBartlomiej Twardowski
 
Information Retrieval Models for Recommender Systems - PhD slides
byDaniel Valcarce
 
Recommender system algorithm and architecture
byLiang Xiang
 
Deep Q-Learning
byNikolay Pavlov
 
Deep Learning in Recommender Systems - RecSys Summer School 2017
byBalázs Hidasi
 
Data Mining: Concepts and techniques classification _chapter 9 :advanced methods
bySalah Amean
 

Similar to [Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

PDF
Fundamentals of Deep Recommender Systems
byWQ Fan
 
PDF
Deep Learning for Recommender Systems @ TDC SP 2019
byGabriel Moreira
 
PDF
Recurrent Neural Networks for Recommendations and Personalization with Nick P...
byDatabricks
 
PDF
Deep Learning Recommender Systems
byCristian Javier Martinez
 
PDF
Transformers4rec: Harnessing NLP Advancements for Cutting-Edge Recommender Sy...
byZilliz
 
PPTX
Deep Learning for Recommender Systems
byNick Pentreath
 
PDF
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...
byClaudio Greco
 
PDF
Heterogeneous semantic graph embedding assisted edge sensitive learning for c...
byIAESIJAI
 
PDF
Silk Data - Review Lecture on Recommendation Systems
byNikolay Karelin
 
PDF
A tensorflow recommending system for news — Fabrício Vargas Matos (Hearst tv)...
byPAPIs.io
 
PDF
Sistemas de Recomendação sem Enrolação
byGabriel Moreira
 
PDF
Deep Recommender Systems - PAPIs.io LATAM 2018
byGabriel Moreira
 
PDF
[MS] Thesis Defense
byHeng-Xiu Xu
 
PDF
News Session-Based Recommendations Using Deep Neural Networks
byFelipe Ferreira
 
PDF
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...
byAlessandro Suglia
 
PDF
Eat it, Review it: A New Approach for Review Prediction
byvivatechijri
 
PDF
Recommending Semantic Nearest Neighbors Using Storm and Dato
byAshok Venkatesan
 
PDF
Icon18revrec sudeshna
byMuthusamy Chelliah
 
PPTX
You Should This! Let me explain to you why.
byFabrizio Silvestri
 
PDF
A domain-independent Framework for building Conversational Recommender Systems
byFedelucio Narducci
 
Fundamentals of Deep Recommender Systems
byWQ Fan
 
Deep Learning for Recommender Systems @ TDC SP 2019
byGabriel Moreira
 
Recurrent Neural Networks for Recommendations and Personalization with Nick P...
byDatabricks
 
Deep Learning Recommender Systems
byCristian Javier Martinez
 
Transformers4rec: Harnessing NLP Advancements for Cutting-Edge Recommender Sy...
byZilliz
 
Deep Learning for Recommender Systems
byNick Pentreath
 
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...
byClaudio Greco
 
Heterogeneous semantic graph embedding assisted edge sensitive learning for c...
byIAESIJAI
 
Silk Data - Review Lecture on Recommendation Systems
byNikolay Karelin
 
A tensorflow recommending system for news — Fabrício Vargas Matos (Hearst tv)...
byPAPIs.io
 
Sistemas de Recomendação sem Enrolação
byGabriel Moreira
 
Deep Recommender Systems - PAPIs.io LATAM 2018
byGabriel Moreira
 
[MS] Thesis Defense
byHeng-Xiu Xu
 
News Session-Based Recommendations Using Deep Neural Networks
byFelipe Ferreira
 
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...
byAlessandro Suglia
 
Eat it, Review it: A New Approach for Review Prediction
byvivatechijri
 
Recommending Semantic Nearest Neighbors Using Storm and Dato
byAshok Venkatesan
 
Icon18revrec sudeshna
byMuthusamy Chelliah
 
You Should This! Let me explain to you why.
byFabrizio Silvestri
 
A domain-independent Framework for building Conversational Recommender Systems
byFedelucio Narducci
 

More from Gabriel Moreira

PDF
Discovering User's Topics of Interest in Recommender Systems
byGabriel Moreira
 
PDF
Feature Engineering - Getting most out of data for predictive models
byGabriel Moreira
 
PDF
Feature Engineering - Getting most out of data for predictive models - TDC 2017
byGabriel Moreira
 
PDF
Introduction to Data Science
byGabriel Moreira
 
PDF
Discovering User's Topics of Interest in Recommender Systems @ Meetup Machine...
byGabriel Moreira
 
PDF
Python for Data Science
byGabriel Moreira
 
PPT
ArcGIS Runtime For Android
byGabriel Moreira
 
PDF
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PDF
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PDF
lsrs15_ciandt
byGabriel Moreira
 
PPTX
An Investigation Of EXtreme Programming Practices
byGabriel Moreira
 
PDF
Python for Data Science - TDC 2015
byGabriel Moreira
 
PDF
CI&T Tech Summit 2017 - Machine Learning para Sistemas de Recomendação
byGabriel Moreira
 
PDF
Python for Data Science - Python Brasil 11 (2015)
byGabriel Moreira
 
PPTX
Using Neural Networks and 3D sensors data to model LIBRAS gestures recognitio...
byGabriel Moreira
 
PPTX
EARLY-FIX: Um Framework para Predição de Manutenção Corretiva de Software uti...
byGabriel Moreira
 
PPTX
Developing GeoGames for Education with Kinect and Android for ArcGIS Runtime
byGabriel Moreira
 
PPTX
Continuous Inspection - An effective approch towards Software Quality Product...
byGabriel Moreira
 
PPT
Agile Testing e outros amendoins
byGabriel Moreira
 
PPTX
Dojo Imagem de Android - 19/06/2012
byGabriel Moreira
 
Discovering User's Topics of Interest in Recommender Systems
byGabriel Moreira
 
Feature Engineering - Getting most out of data for predictive models
byGabriel Moreira
 
Feature Engineering - Getting most out of data for predictive models - TDC 2017
byGabriel Moreira
 
Introduction to Data Science
byGabriel Moreira
 
Discovering User's Topics of Interest in Recommender Systems @ Meetup Machine...
byGabriel Moreira
 
Python for Data Science
byGabriel Moreira
 
ArcGIS Runtime For Android
byGabriel Moreira
 
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...
byGabriel Moreira
 
lsrs15_ciandt
byGabriel Moreira
 
An Investigation Of EXtreme Programming Practices
byGabriel Moreira
 
Python for Data Science - TDC 2015
byGabriel Moreira
 
CI&T Tech Summit 2017 - Machine Learning para Sistemas de Recomendação
byGabriel Moreira
 
Python for Data Science - Python Brasil 11 (2015)
byGabriel Moreira
 
Using Neural Networks and 3D sensors data to model LIBRAS gestures recognitio...
byGabriel Moreira
 
EARLY-FIX: Um Framework para Predição de Manutenção Corretiva de Software uti...
byGabriel Moreira
 
Developing GeoGames for Education with Kinect and Android for ArcGIS Runtime
byGabriel Moreira
 
Continuous Inspection - An effective approch towards Software Quality Product...
byGabriel Moreira
 
Agile Testing e outros amendoins
byGabriel Moreira
 
Dojo Imagem de Android - 19/06/2012
byGabriel Moreira
 

Recently uploaded

PDF
December Patch Tuesday
byIvanti
 
PDF
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
PDF
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
DOCX
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
PDF
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
PPTX
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
PDF
Energy Storage Landscape Clean Energy Ministerial
bySurajitBanerjee38
 
PPTX
Conversational Agents – Building Intelligent Assistants [Virtual Hands-on Wor...
byUiPathCommunity
 
PPTX
Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]
byUiPathCommunity
 
PDF
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
PDF
Making Sense of Raster: From Bit Depth to Better Workflows
bySafe Software
 
PDF
Cybersecurity: Safeguarding Digital Assets
byYasir Naveed Riaz
 
PDF
The year in review - MarvelClient in 2025
bypanagenda
 
PPTX
AI's Impact on Cybersecurity - Challenges and Opportunities
byYasir Naveed Riaz
 
PPTX
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
PPTX
Cybersecurity Best Practices - Step by Step guidelines
byYasir Naveed Riaz
 
PDF
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 
PPTX
Chapter 3 Introduction to number system.pptx
byGetachewAbera9
 
PPTX
Cybercrime in the Digital Age: Risks, Impact & Protection
byYasir Naveed Riaz
 
PDF
Is It Possible to Have Wi-Fi Without an Internet Provider
bySidra Jefferi
 
December Patch Tuesday
byIvanti
 
Real-Time Data Insight Using Microsoft Forms for Business
byElevate
 
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
Energy Storage Landscape Clean Energy Ministerial
bySurajitBanerjee38
 
Conversational Agents – Building Intelligent Assistants [Virtual Hands-on Wor...
byUiPathCommunity
 
Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]
byUiPathCommunity
 
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
Making Sense of Raster: From Bit Depth to Better Workflows
bySafe Software
 
Cybersecurity: Safeguarding Digital Assets
byYasir Naveed Riaz
 
The year in review - MarvelClient in 2025
bypanagenda
 
AI's Impact on Cybersecurity - Challenges and Opportunities
byYasir Naveed Riaz
 
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
Cybersecurity Best Practices - Step by Step guidelines
byYasir Naveed Riaz
 
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 
Chapter 3 Introduction to number system.pptx
byGetachewAbera9
 
Cybercrime in the Digital Age: Risks, Impact & Protection
byYasir Naveed Riaz
 
Is It Possible to Have Wi-Fi Without an Internet Provider
bySidra Jefferi
 

[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

  • 1.
    CHAMELEON: A DeepLearning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019
  • 2.
    Introduction "We are leavingthe Information Age and entering the Recommendation Age.". Cris Anderson, "The long tail" 2
  • 3.
    News consumption isthe majority of web traffic (TREVISIOL et al. , 2014b) 3 Introduction
  • 4.
    4 Research trends onNews Recommender Systems (KARIMI et al. , 2018) Introduction
  • 5.
    5 Research trends onNews Recommender Systems (KHAN et al., 2019) Introduction
  • 6.
    6 1. Preferences shift Introduction NewsRecommender Systems Challenges
  • 7.
    7 1. Preferences shift 2.Sparse user profiles Introduction News Recommender Systems Challenges # sessions by user (G1 dataset, 16 days) Q3 = 3 sessions Q2 = 2 sessions Q1 = 1 session
  • 8.
    8 1. Preferences shift 2.Sparse user profiles 3. Fast growing number of items Introduction News Recommender Systems Challenges # fresh articles by day (G1 dataset)
  • 9.
    9 1. Preferences shift 2.Sparse user profiles 3. Fast growing number of items 4. Accelerated item’s value decay Introduction News Recommender Systems Challenges Permanent User and Item Cold-Start Problem Articles age at click time distribution (G1 dataset) Percentile of # clicks Article Age 10% < 4 hours 25% < 5 hours 50% < 8 hours 75% < 14 hours 90% < 26 hours
  • 10.
    Objectives 10 1. To investigatethe quality of news recommendation using session-based algorithms 2. To design a Deep Learning meta-architecture for hybrid and contextual recommendation, based on Recurrent Neural Networks, to improve the quality of recommendations provided by news portals.
  • 11.
    Research Questions ● Howdoes the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 11 ● What is the effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? ● What is the effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? ● Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? ● Is the proposed approach effective to balance the competing quality factors of accuracy and novelty? 1 2 3 4 5
  • 12.
    Scope (1/2) ● Session-basednews recommendation ● Task: Next-click prediction for user sessions in a news portal 12
  • 13.
    Scope (1/3) ● Session-basednews recommendation ● Task: Next-click prediction for user sessions in a news portal 13 User session clicks C1 C2 C3 C4 Next-click prediction model Article B Article A Article C Article D ... Ranked articles Recommendable articles (sampled)
  • 14.
    Scope (2/3) ● Theproposed Deep Learning meta-architecture should be: ○ Session-based ○ Hybrid ○ Contextual 14 ● Recommendation quality factors to be evaluated: ○ Accuracy ○ Item Coverage ○ Novelty ○ Diversity ○ Robustness against the Item Cold-Start Problem
  • 15.
    Recommender System Content-based filteringCollaborative filtering Model-based filteringMemory-based filtering Item-basedUser-based ML-based: Clustering, Association Rules, Matrix Factorization, Neural Networks Hybrid filtering+ = Focus of this research 15 Scope (3/3)
  • 16.
  • 17.
    Deep Learning adoptionin RecSys 2007 2015 2016 2017-2019 Deep Boltzmann Machines for rating prediction calm before the storm A few seminal papers First DLRS workshop and papers on RecSys, KDD, SIGIR Continued increase 17
  • 18.
    Research directions inDL-RecSys And their combinations... Session-based recommendations with RNNs Feature Extraction directly from the content Learning Item embeddings Deep Collaborative Filtering Multi-view Learning Focus of this research 18
  • 19.
    Related work 19 Main inspirations: ●GRU4Rec (HIDASI, 2016) - The seminal work on the usage of Recurrent Neural Networks (RNN) on session-based recommendations, and subsequent work (Hidasi,2017). ● MV-DNN (ELKAHKI, 2015) - Adapted Deep Structured Semantic Model (DSSM) for the recommendation task. MV-DNN maps users and items to a latent space, where the cosine similarity between users and their preferred items is maximized. That approach makes it possible to keep the neural network architecture static, rather than adding new units into the output layer for each new item.
  • 20.
    CHAMELEON: A DeepLearning Meta-Architecture for News Recommendation
  • 21.
    The conceptual modelof news relevance factors 2121 News relevance Topics Entities Publisher News static properties Recency Popularity News dynamic properties News article User TimeLocation Device User current context Long-term interests Short-term interests Global factors Season- ality User interests Breaking events Popular Topics Referrer
  • 22.
    Meta-Architecture Requirements (1/2) ●RR1 - to provide personalized news recommendations in extreme cold-start scenarios, as most news are fresh and most users cannot be identified ● RR2 - to automatically learn news representations from textual content and news metadata, minimizing the need of manual feature engineering ● RR3 - to leverage the user session information, as the sequence of interacted news may indicate user's short-term preferences for session-based recommendations ● RR4 - to leverage users' contextual information, as a rich data source in such information scarcity about the user 22
  • 23.
    Meta-Architecture Requirements (2/2) ●RR5 - to model explicitly contextual news properties (popularity and recency), as those are important factors on news interest life cycle ● RR6 - to support an increasing number of new items and users by incremental model retraining (online learning), without the need to retrain on the whole historical dataset ● RR7 - to provide a modular structure for news recommendation, allowing its modules to be instantiated by different and increasingly advanced neural network architectures and methods 23
  • 24.
    The CHAMELEON A masteron context adaptation 24
  • 25.
    The CHAMELEON Each eyecan pivot independently, allowing to focus two different objects simultaneously. 25
  • 26.
  • 27.
    The CHAMELEON Meta-Architecturefor News RS 27 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings
  • 28.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Representation (ACR) module Responsible for learning distributed representations (embeddings) from news’ contents. Sub-modules: ● Textual Features Representation (TFR) ● Content Embeddings Training (CET) 28 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings
  • 29.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 29 Inputs Article text represented by Word embeddings Word embeddings pre-trained in larger corpus: ● Word2Vec ● GloVe Article Content Embeddings When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Article Content Embedding (ACE)
  • 30.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Textual Features Representation (TFR) sub-module ● Learns relevant features from article textual content (RR2). ● Can be instantiated by a Deep NLP architecture like: ○ CNN ○ RNN ○ QRNN ○ Attention mechanisms 30 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Embedding (ACE) Content Embeddings Training (CET)
  • 31.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Embedding ● The learned article distributed representation ● After training, it is stored in a repository for further usage by NAR module 31 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Content Embeddings Training (CET)
  • 32.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Content Embeddings Training (CET) ● Responsible to train Article Content Embeddings. ○ Supervised task: e.g. predict article metadata attributes ○ Unsupervised task: e.g. reconstruct a compressed version of the input (autoencoder) ● May be instantiated as a sequence of feed-forward Fully Connected layers. 32 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings
  • 33.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Next-Article Recommendation (NAR) module ● Provide news articles recommendations for each interaction (I) in active user sessions. ● For each recommendation request, NAR module generates a ranked list of the most likely articles user might read in a given session. Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) Time Location Device User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session 33
  • 34.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 1 - Article Content Embedding Pre-trained by the ACR module (RR2) 2 - User Context Properties of user current context (RR4): ○ Time (Period of day, Weekday / Weekend, Month, Quarter) ○ Location (City, State, Country) ○ Device (eg. Desktop / Smartphone) ○ Device OS (eg. Android / iOS) 34
  • 35.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 3 - Article context Contextual properties of the article (RR5): ● Recent Popularity - Popularity of the article within the last hour ● Recency - Time elapsed since the article was published 35
  • 36.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 36 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Contextual Article Representation (CAR) sub-module ● Merges the inputs – article content, article context and user context – and generates the User-Personalized Contextual Article Embedding ● This sub-module might be instantiated as Fully Connected (FC) layers or using Factorization Machines (FM), for example.
  • 37.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 37 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sessions in a batch I1,1 I1,2 I1,3 I1,4 I1,5 I2,1 I2,2 I3,1 I3,2 I3,3
  • 38.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 38 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sampling strategy Negative samples: Articles read by other users in the last hour Positive sample: Next article read by the user in his session Samples
  • 39.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: SEssion Representation (SER) sub-module ● Models short-term user’s preferences in their active sessions (RR3) ● Must be instantiated by an architecture that is able to model the sequence of user clicks, such as RNN, CNN, attention mechanisms. 39 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER)
  • 40.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Predicted Next-Article Embedding ● Represents the expected representation of a news content the user would like to read next in the active session. 40 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding
  • 41.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module ● Recommends articles to a user session based on embeddings similarity, supporting unseen items during inference (RR6). ● The loss function maximizes the similarity between “Predicted Next-Article Embedding” and the “User-Personalized Contextual Article Embedding” corresponding to the positive sample, and minimize similarities with negative samples. 41 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommendations Ranking (RR)
  • 42.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 42 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 1 - Relevance Score of an item for a user session Eq. 2 - Matching function Eq. 3 - Softmax over Relevance Score Eq. 4 - Accuracy Loss function
  • 43.
    The CHAMELEON Meta-Architecturefor News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 43 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 5 - Novelty Loss function Eq. 6 - Novelty Eq. 7 - Recent Normalized Popularity Eq. 8 - Multi-objective Loss Function
  • 44.
    The CHAMELEON Meta-Architecturefor News RS 44 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings
  • 45.
    45 Comparison among DeepArchitectures for News RS Characteristics (SONG, 2016) (KUMAR, 2017) (PARK, 2017) (OKURA, 2017) (WANG, 2018) (ZHANG, 2018) CHAMELEON Session-based Multiple quality objectives Temporal articles relevance decay (exponential function) (decay learned by the model) Time-aware negative sampling N/A - Does not use negative sampling Article Context (Recency, Recent Pop) User Context Textual Content Features Letter trigrams, processed by MLP doc2vec PV-DBoW Denoising auto-encoder Entities enrichment of title with a Knowledge Graph Character-level CNN Sup. and Unsup. ACR instantiations (CNN, GRU) DNN architecture for sequence modeling RNN RNN RNN RNN DNN with attention RNN RNN Loss function Listwise ranking based on DSSM Listwise ranking based on DSSM Pairwise ranking BRP, TOP1 Pairwise ranking with position bias Pointwise ranking: Log loss Pairwise ranking: BRP, TOP1 Listwise ranking based on DSSM (custom)
  • 46.
  • 47.
    ACR module -Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Output: Article Metadata Attributes ● If more than one attribute is available for classification, Multi-Task Learning (MTL) is used. TASK A TASK B TASK C Task-specific layers Shared layers 47 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers When a news article is published...
  • 48.
    ACR module -Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 48 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers Output: Article Metadata Attributes ● The last layer of ACR module is responsible to classify one or more article metadata attributes ● Each attribute might be single-label (e.g. category) or multi-label (e.g. tags), in Eq. 9 and 10, respectively Eq. 9 - Softmax Eq. 10 - Sigmoid Eq. 11 - Cross-entropy When a news article is published...
  • 49.
    ACR module -Supervised - RNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 49 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers GRU encoderinput word embeddings hidden state RNN based of TFR sub-module ● Process the sequence of word embedding using an RNN ● P.s. Not as computationally efficient as CNNs
  • 50.
    ACR module -Unsupervised Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 50 Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) GRU encoder Fully Connected layer Reconstructed word embeddings Fully Connected layer input word embeddings hidden state GRU decoder New York is a multicultural city , ... hidden state Article Content Embeddings Word Embeddings Autoencoder based instantiation of the ACR module ● A Sequence Denoising Autoencoder ● The objective is to reconstruct the sequence of corrupted input word embeddings from a compressed representation (Article Content Embedding) ● P.s. The sequence of input words is reversed to improve reconstruction accuracy
  • 51.
    51 NAR module instantiation Article Context Article Content Embeddings Next-ArticleRecommendation (NAR) User context past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active articleActive Sessions Predicted Next-Article Embedding Session Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Fully Connected layers UGRNN + Fully Connected layers Time Location Device Referrer User interaction When a user reads a news article... Active User Session NAR module instantiation ● Learnable parameters for centering and scale input features ● CAR sub-module was instantiated by 2 FC layers (Leaky ReLU + tanh) ● SER sub-module was instantiated by a 2-layer UGRNN cell (COLLINS, 2017) followed by 2 FC layers (Leaky ReLU + tanh).
  • 52.
    CHAMELEON Implementation 52 ● CHAMELEON’sinstantiations are implemented using TensorFlow https://github.com/gabrielspmoreira/chameleon_recsys ● Training and evaluation performed in Google Cloud Platform ML Engine
  • 53.
  • 54.
    Evaluation Protocol 54 Task: Foreach item within a session, predict the next-clicked item from a set composed by the positive sample (correct article) and 50 negative samples. Hours Input I1 I2 I3 I4 I2 I3 I4 I5 Expected Output (labels)
  • 55.
    Metrics (1/2) 55 Accuracy HR@10Hit Rate - Checks whether the positive item is among the top-N ranked items MRR@10 Mean Reciprocal Rank - Ranking metric which assigns higher scores at top ranks Item Coverage COV@10 # recommended articles @ top-n # recommendable articles
  • 56.
  • 57.
  • 58.
    Datasets (1/2) 58 G1 dataset ●Provided by Globo.com, the most popular news portal in Brazil ● Made publicly available by this research for the community https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom
  • 59.
    59 G1 (globo.com) Adressa LanguagePortuguese Norwegian Period (days) 16 16 # users 322 k 314 k # sessions 1,048 k 982 k # clicks 2,988 k 2,648 k # articles 46 k 13 k Avg. session length * 2.84 2.70 * After removing single-click sessions and sessions with more than 20 clicks. Datasets (2/2)
  • 60.
    Baseline algorithms 60 Neural methods GRU4RecA landmark neural architecture using RNNs for session-based recommendation SR-GNN A recently state-of-the-art architecture for session-based recommendation based on GNN. Frequent patterns methods CO A simple method of co-occurrence based on association rules of length two. SR Sequential rules of size two. kNN methods Item-kNN Most similar articles to the last read one, in terms of the cosine similarity between the vector of their occurrence in sessions. V-SkNN Compares the entire active session with past (neighboring) sessions to determine items to be recommended. The similarity function emphasizes items that appear later within the session. Non-personalized methods RP Recently Popular - Most viewed articles during the last hour. CB Content-Based - Most similar articles (content embedding) to the last user click.
  • 61.
  • 62.
    Research Questions How doesthe proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality factors (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 62 DLRS @ RecSys, 2018 IEEE Access, 2019 (Qualis A2, Eng. IV) 1
  • 63.
    G1 dataset +14.2% +19.6% News recommendationquality of session-based algorithms 1 63 Other neural-based methods
  • 64.
  • 65.
    65 Average MRR@10 byhour (evaluation each 5 hours), during 16 days G1 dataset News recommendation quality of session-based algorithms 1
  • 66.
    66 Average MRR@10 byhour (evaluation each 5 hours), during 16 days Adressa dataset News recommendation quality of session-based algorithms 1
  • 67.
    67 Recommendation Accuracy (HR@10) of algorithms (lines) x Avg. Normalized Popularity (bars) by session click order Adressa dataset News recommendation quality of session-based algorithms 1 G1 dataset
  • 68.
    News recommendation qualityof session-based algorithms 68 Evaluation of other quality factors G1 dataset 1
  • 69.
    News recommendation qualityof session-based algorithms 69 Evaluation of other quality factors 1Adressa dataset
  • 70.
    Research Questions 70 What isthe effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? IEEE Access, 2019 (Qualis A2, Eng. IV) 2
  • 71.
    71 The effect ofdifferent types of information 2 Input Configuration Feature Sets IC1 Article Id IC2 IC1 + Article Context (Novelty and Recency) IC3 IC2 + the Article Content Embeddings (ACE) learned by the supervised instantiation of the ACR module IC4 IC3 + Article Metadata IC5 IC4 + User Context
  • 72.
    72 The effect ofdifferent types of information 2
  • 73.
    Research Questions 73 What isthe effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? INRA @ RecSys, 2019 3
  • 74.
    74 The effect ofdifferent content representations 3ACRmodule instantiations
  • 75.
    75 Recommender HR@10 MRR@10 No-ACE0.6281 0.3066 Supervised CNN 0.6585 0.3395 GRU 0.6585 0.3388 Unsupervised SDA-GRU 0.6418 0.3160 W2V*TF-IDF 0.6575 0.3291 LSA 0.6686* 0.3423 doc2vec 0.6368 0.3119 Comparison of different content encodings for CHAMELEON The effect of different content representations G1 dataset 3
  • 76.
    76Comparison of differentcontent encodings for CHAMELEON The effect of different content representations Adressa dataset 3 Recommender HR@10 MRR@10 No-ACE 0.6816 0.3252 Supervised CNN 0.6860 0.3333 GRU 0.6856 0.3327 Unsupervised SDA-GRU 0.6905 0.3360 W2V*TF-IDF 0.6913 0.3402 LSA 0.6935 0.3403 doc2vec 0.6898 0.3402
  • 77.
    Research Questions Is theproposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? 77 4
  • 78.
    78 The effect onitem cold-start 4 # Batches before First Recommendation (BFR@n) G1 dataset
  • 79.
    79 The effect onitem cold-start 4 Adressa dataset # Batches before First Recommendation (BFR@n)
  • 80.
    Research Questions Is theproposed approach effective to balance the competing quality factors of accuracy and novelty? 80 IEEE Access, 2019 (Qualis A2, Eng. IV) 5
  • 81.
    81 Balancing the trade-offbetween Accuracy and Novelty 5 G1 dataset Adressa dataset
  • 82.
  • 83.
    Main Contributions 83 1. Theinvestigation of the stated research questions related to the effectiveness of session-based recommendation algorithms on the news domain 2. The design, implementation, and evaluation of a Deep Learning meta-architecture – the CHAMELEON – for hybrid and contextual session-based news recommendation
  • 84.
    Complementary Contributions 84 1. Theelaboration of a Conceptual Model of factors that affect news relevance 2. The four instantiations of ACR and NAR modules of CHAMELEON 3. The open-source implementation of CHAMELEON 1 4. The temporal offline evaluation protocol for news recommendation 5. The adaptation of rank- and relevance-sensitive novelty and diversity metrics Additional Contribution 6. The preprocessing and sharing of two novel datasets for evaluation of hybrid and contextual RS for news (G1) 2 and articles (CI&T Desktop) 3 1 https://github.com/gabrielspmoreira/chameleon_recsys 2 https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom 3 https://www.kaggle.com/gspmoreira/articles-sharing-reading-from-cit-deskdrop
  • 85.
    Recommendations 85 1. Online evaluationof CHAMELEON in a live news portal 2. To balance more than two quality factors (including diversity) 3. To adapt CHAMELEON for session-aware news recommendation 4. To explore different negative sample approaches (e.g., by geographic region) 5. Explore other Deep NLP techniques (e.g. BERT) to produce better article content embeddings 6. To deepen the investigation of the RQ4 about the item cold-start problem (next paper)
  • 86.
    Suggestions for FutureWorks 86 1. Instantiate the SER sub-module with a neural network architecture other than RNNs (e.g., CNN, QRNN, GNN) 2. Investigate attention mechanisms as a way to improve accuracy and provide recommendation explanations 3. Investigate “noisy outliers” in the user profiles 4. Adapt CHAMELEON for contextual and hybrid session-based recommendations in other domains like e-commerce, entertainment and media
  • 87.
    Published Articles 87 1. MOREIRA,Gabriel de Souza Pereira; SOUZA JUNIOR, G. J. A. ; CUNHA, A. M. . Comparing offline and online recommender system evaluations on long-tail distributions. In: ACM Recommender Systems Conference, 2015, Viena, Austria. Poster Proceedings of ACM RecSys 2015, 2015. v. 1441. p. 2. MOREIRA, GABRIEL DE SOUZA P.; SOUZA, GILMAR . A Recommender System to tackle Enterprise Collaboration. In: the 10th ACM Conference, 2016, Boston. Proceedings of the 10th ACM Conference on Recommender Systems - RecSys '16 (Demo paper). New York: ACM Press, 2016. p. 395. DOI https://doi.org/10.1145/2959100.2959115 3. MOREIRA, GABRIEL. CHAMELEON: a Deep Learning Meta-Architecture for News Recommender Systems. In: 12th ACM Conference on Recommender Systems - RecSys'18, 2018, Vancouver. Proceedings of the 12th ACM Conference on Recommender Systems RecSys'18 - Doctoral Symposium. New York: ACM Press, 2018. DOI https://doi.org/10.1145/3240323.3240331 4. MOREIRA, GABRIEL; FERREIRA, FELIPE ; DA CUNHA, ADILSON MARQUES . News Session-Based Recommendations using Deep Neural Networks. Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems - DLRS 2018. New York: ACM Press, 2018. p.15. DOI https://doi.org/10.1145/3270323.3270328 5. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES . On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems. Proceedings of the 7th Workshop on News Recommendation and Analytics (INRA 2019), in conjunction with RecSys 2019, September 19, 2019, Copenhagen 6. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES .. "Contextual Hybrid Session-Based News Recommendation With Recurrent Neural Networks." IEEE Access 7 (2019): 169185-169203.
  • 88.
    CHAMELEON: A DeepLearning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019