[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

1. CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019

2. Introduction "We are leaving the Information Age and entering the Recommendation Age.". Cris Anderson, "The long tail" 2

3. News consumption is the majority of web traffic (TREVISIOL et al. , 2014b) 3 Introduction

4. 4 Research trends on News Recommender Systems (KARIMI et al. , 2018) Introduction

5. 5 Research trends on News Recommender Systems (KHAN et al., 2019) Introduction

6. 6 1. Preferences shift Introduction News Recommender Systems Challenges

7. 7 1. Preferences shift 2. Sparse user profiles Introduction News Recommender Systems Challenges # sessions by user (G1 dataset, 16 days) Q3 = 3 sessions Q2 = 2 sessions Q1 = 1 session

8. 8 1. Preferences shift 2. Sparse user profiles 3. Fast growing number of items Introduction News Recommender Systems Challenges # fresh articles by day (G1 dataset)

9. 9 1. Preferences shift 2. Sparse user profiles 3. Fast growing number of items 4. Accelerated item’s value decay Introduction News Recommender Systems Challenges Permanent User and Item Cold-Start Problem Articles age at click time distribution (G1 dataset) Percentile of # clicks Article Age 10% < 4 hours 25% < 5 hours 50% < 8 hours 75% < 14 hours 90% < 26 hours

10. Objectives 10 1. To investigate the quality of news recommendation using session-based algorithms 2. To design a Deep Learning meta-architecture for hybrid and contextual recommendation, based on Recurrent Neural Networks, to improve the quality of recommendations provided by news portals.

11. Research Questions ● How does the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 11 ● What is the effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? ● What is the effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? ● Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? ● Is the proposed approach effective to balance the competing quality factors of accuracy and novelty? 1 2 3 4 5

12. Scope (1/2) ● Session-based news recommendation ● Task: Next-click prediction for user sessions in a news portal 12

13. Scope (1/3) ● Session-based news recommendation ● Task: Next-click prediction for user sessions in a news portal 13 User session clicks C1 C2 C3 C4 Next-click prediction model Article B Article A Article C Article D ... Ranked articles Recommendable articles (sampled)

14. Scope (2/3) ● The proposed Deep Learning meta-architecture should be: ○ Session-based ○ Hybrid ○ Contextual 14 ● Recommendation quality factors to be evaluated: ○ Accuracy ○ Item Coverage ○ Novelty ○ Diversity ○ Robustness against the Item Cold-Start Problem

15. Recommender System Content-based filtering Collaborative filtering Model-based filteringMemory-based filtering Item-basedUser-based ML-based: Clustering, Association Rules, Matrix Factorization, Neural Networks Hybrid filtering+ = Focus of this research 15 Scope (3/3)

16. Deep Learning for Recommender Systems

17. Deep Learning adoption in RecSys 2007 2015 2016 2017-2019 Deep Boltzmann Machines for rating prediction calm before the storm A few seminal papers First DLRS workshop and papers on RecSys, KDD, SIGIR Continued increase 17

18. Research directions in DL-RecSys And their combinations... Session-based recommendations with RNNs Feature Extraction directly from the content Learning Item embeddings Deep Collaborative Filtering Multi-view Learning Focus of this research 18

19. Related work 19 Main inspirations: ● GRU4Rec (HIDASI, 2016) - The seminal work on the usage of Recurrent Neural Networks (RNN) on session-based recommendations, and subsequent work (Hidasi,2017). ● MV-DNN (ELKAHKI, 2015) - Adapted Deep Structured Semantic Model (DSSM) for the recommendation task. MV-DNN maps users and items to a latent space, where the cosine similarity between users and their preferred items is maximized. That approach makes it possible to keep the neural network architecture static, rather than adding new units into the output layer for each new item.

20. CHAMELEON: A Deep Learning Meta-Architecture for News Recommendation

21. The conceptual model of news relevance factors 2121 News relevance Topics Entities Publisher News static properties Recency Popularity News dynamic properties News article User TimeLocation Device User current context Long-term interests Short-term interests Global factors Season- ality User interests Breaking events Popular Topics Referrer

22. Meta-Architecture Requirements (1/2) ● RR1 - to provide personalized news recommendations in extreme cold-start scenarios, as most news are fresh and most users cannot be identiﬁed ● RR2 - to automatically learn news representations from textual content and news metadata, minimizing the need of manual feature engineering ● RR3 - to leverage the user session information, as the sequence of interacted news may indicate user's short-term preferences for session-based recommendations ● RR4 - to leverage users' contextual information, as a rich data source in such information scarcity about the user 22

23. Meta-Architecture Requirements (2/2) ● RR5 - to model explicitly contextual news properties (popularity and recency), as those are important factors on news interest life cycle ● RR6 - to support an increasing number of new items and users by incremental model retraining (online learning), without the need to retrain on the whole historical dataset ● RR7 - to provide a modular structure for news recommendation, allowing its modules to be instantiated by different and increasingly advanced neural network architectures and methods 23

24. The CHAMELEON A master on context adaptation 24

25. The CHAMELEON Each eye can pivot independently, allowing to focus two different objects simultaneously. 25

26. The CHAMELEON An accurate hunter 26

27. The CHAMELEON Meta-Architecture for News RS 27 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings

28. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Representation (ACR) module Responsible for learning distributed representations (embeddings) from news’ contents. Sub-modules: ● Textual Features Representation (TFR) ● Content Embeddings Training (CET) 28 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings

29. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 29 Inputs Article text represented by Word embeddings Word embeddings pre-trained in larger corpus: ● Word2Vec ● GloVe Article Content Embeddings When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Article Content Embedding (ACE)

30. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Textual Features Representation (TFR) sub-module ● Learns relevant features from article textual content (RR2). ● Can be instantiated by a Deep NLP architecture like: ○ CNN ○ RNN ○ QRNN ○ Attention mechanisms 30 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Word Embeddings Article Content Embedding (ACE) Content Embeddings Training (CET)

31. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Article Content Embedding ● The learned article distributed representation ● After training, it is stored in a repository for further usage by NAR module 31 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Content Embeddings Training (CET)

32. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Content Embeddings Training (CET) ● Responsible to train Article Content Embeddings. ○ Supervised task: e.g. predict article metadata attributes ○ Unsupervised task: e.g. reconstruct a compressed version of the input (autoencoder) ● May be instantiated as a sequence of feed-forward Fully Connected layers. 32 Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings

33. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Next-Article Recommendation (NAR) module ● Provide news articles recommendations for each interaction (I) in active user sessions. ● For each recommendation request, NAR module generates a ranked list of the most likely articles user might read in a given session. Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) Time Location Device User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session 33

34. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 1 - Article Content Embedding Pre-trained by the ACR module (RR2) 2 - User Context Properties of user current context (RR4): ○ Time (Period of day, Weekday / Weekend, Month, Quarter) ○ Location (City, State, Country) ○ Device (eg. Desktop / Smartphone) ○ Device OS (eg. Android / iOS) 34

35. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Active User Session Time Location Device Article Content Embedding Popularity Recency Inputs 3 - Article context Contextual properties of the article (RR5): ● Recent Popularity - Popularity of the article within the last hour ● Recency - Time elapsed since the article was published 35

36. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 36 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Contextual Article Representation (CAR) sub-module ● Merges the inputs – article content, article context and user context – and generates the User-Personalized Contextual Article Embedding ● This sub-module might be instantiated as Fully Connected (FC) layers or using Factorization Machines (FM), for example.

37. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 37 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sessions in a batch I1,1 I1,2 I1,3 I1,4 I1,5 I2,1 I2,2 I3,1 I3,2 I3,3

38. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 38 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding Sampling strategy Negative samples: Articles read by other users in the last hour Positive sample: Next article read by the user in his session Samples

39. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: SEssion Representation (SER) sub-module ● Models short-term user’s preferences in their active sessions (RR3) ● Must be instantiated by an architecture that is able to model the sequence of user clicks, such as RNN, CNN, attention mechanisms. 39 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER)

40. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Predicted Next-Article Embedding ● Represents the expected representation of a news content the user would like to read next in the active session. 40 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommendations Ranking (RR) Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding

41. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module ● Recommends articles to a user session based on embeddings similarity, supporting unseen items during inference (RR6). ● The loss function maximizes the similarity between “Predicted Next-Article Embedding” and the “User-Personalized Contextual Article Embedding” corresponding to the positive sample, and minimize similarities with negative samples. 41 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Recommended articles Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommendations Ranking (RR)

42. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 42 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 1 - Relevance Score of an item for a user session Eq. 2 - Matching function Eq. 3 - Softmax over Relevance Score Eq. 4 - Accuracy Loss function

43. The CHAMELEON Meta-Architecture for News RS Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Recommendations Ranking (RR) sub-module 43 Dynamic Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context User interaction past read articles Article context candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Active User Session Time Location Device Article Content Embedding Popularity Recency Contextual Article Representation (CAR) User-Personalized Contextual Article Embedding SEssion Representation (SER) Predicted Next-Article Embedding Recommended articles Recommendations Ranking (RR) Eq. 5 - Novelty Loss function Eq. 6 - Novelty Eq. 7 - Recent Normalized Popularity Eq. 8 - Multi-objective Loss Function

44. The CHAMELEON Meta-Architecture for News RS 44 Dynamic Article Context Article Content Embeddings Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) Next-Article Recommendation (NAR) Time Location Device When a news article is published... User context User interaction past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active article Active Sessions When a user reads a news article... Predicted Next-Article Embedding SEssion Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Content word embeddings New York is a multicultural city , ... News Article Active User Session Module Sub-Module EmbeddingInput Output Data repositoryAttributes Article Content Embedding (ACE) Legend: Word Embeddings

45. 45 Comparison among Deep Architectures for News RS Characteristics (SONG, 2016) (KUMAR, 2017) (PARK, 2017) (OKURA, 2017) (WANG, 2018) (ZHANG, 2018) CHAMELEON Session-based Multiple quality objectives Temporal articles relevance decay (exponential function) (decay learned by the model) Time-aware negative sampling N/A - Does not use negative sampling Article Context (Recency, Recent Pop) User Context Textual Content Features Letter trigrams, processed by MLP doc2vec PV-DBoW Denoising auto-encoder Entities enrichment of title with a Knowledge Graph Character-level CNN Sup. and Unsup. ACR instantiations (CNN, GRU) DNN architecture for sequence modeling RNN RNN RNN RNN DNN with attention RNN RNN Loss function Listwise ranking based on DSSM Listwise ranking based on DSSM Pairwise ranking BRP, TOP1 Pairwise ranking with position bias Pointwise ranking: Log loss Pairwise ranking: BRP, TOP1 Listwise ranking based on DSSM (custom)

46. CHAMELEON Architecture Instatiations

47. ACR module - Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: Output: Article Metadata Attributes ● If more than one attribute is available for classification, Multi-Task Learning (MTL) is used. TASK A TASK B TASK C Task-specific layers Shared layers 47 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers When a news article is published...

48. ACR module - Supervised - CNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 48 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Convolutional Neural Network (CNN) conv-3 (128) max-pooling conv-4 (128) max-pooling conv-5 (128) max-pooling Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers Output: Article Metadata Attributes ● The last layer of ACR module is responsible to classify one or more article metadata attributes ● Each attribute might be single-label (e.g. category) or multi-label (e.g. tags), in Eq. 9 and 10, respectively Eq. 9 - Softmax Eq. 10 - Sigmoid Eq. 11 - Cross-entropy When a news article is published...

49. ACR module - Supervised - RNN-based Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 49 Article Content Embeddings Article Content Representation (ACR)Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) Word Embeddings Fully Connected layers Category Target Article Metadata Attributes Fully Connected layers GRU encoderinput word embeddings hidden state RNN based of TFR sub-module ● Process the sequence of word embedding using an RNN ● P.s. Not as computationally efficient as CNNs

50. ACR module - Unsupervised Module Sub-Module EmbeddingInput Output Data repositoryAttributesLegend: 50 Article Content Representation (ACR) Textual Features Representation (TFR) Content Embeddings Training (CET) When a news article is published... Content word embeddings New York is a multicultural city , ... News Article Article Content Embedding (ACE) GRU encoder Fully Connected layer Reconstructed word embeddings Fully Connected layer input word embeddings hidden state GRU decoder New York is a multicultural city , ... hidden state Article Content Embeddings Word Embeddings Autoencoder based instantiation of the ACR module ● A Sequence Denoising Autoencoder ● The objective is to reconstruct the sequence of corrupted input word embeddings from a compressed representation (Article Content Embedding) ● P.s. The sequence of input words is reversed to improve reconstruction accuracy

51. 51 NAR module instantiation Article Context Article Content Embeddings Next-Article Recommendation (NAR) User context past read articles Popularity Recency Article context Article Content Embedding candidate next articles (positive and neg.) active articleActive Sessions Predicted Next-Article Embedding Session Representation (SER) Recommendations Ranking (RR) User-Personalized Contextual Article Embedding Recommended articles Contextual Article Representation (CAR) Fully Connected layers UGRNN + Fully Connected layers Time Location Device Referrer User interaction When a user reads a news article... Active User Session NAR module instantiation ● Learnable parameters for centering and scale input features ● CAR sub-module was instantiated by 2 FC layers (Leaky ReLU + tanh) ● SER sub-module was instantiated by a 2-layer UGRNN cell (COLLINS, 2017) followed by 2 FC layers (Leaky ReLU + tanh).

52. CHAMELEON Implementation 52 ● CHAMELEON’s instantiations are implemented using TensorFlow https://github.com/gabrielspmoreira/chameleon_recsys ● Training and evaluation performed in Google Cloud Platform ML Engine

53. Experiments Design

54. Evaluation Protocol 54 Task: For each item within a session, predict the next-clicked item from a set composed by the positive sample (correct article) and 50 negative samples. Hours Input I1 I2 I3 I4 I2 I3 I4 I5 Expected Output (labels)

55. Metrics (1/2) 55 Accuracy HR@10 Hit Rate - Checks whether the positive item is among the top-N ranked items MRR@10 Mean Reciprocal Rank - Ranking metric which assigns higher scores at top ranks Item Coverage COV@10 # recommended articles @ top-n # recommendable articles

56. Metrics (2/2) Novelty ESI-R@10 ESI-RR@10 56

57. Metrics (2/2) 57 Novelty ESI-R@10 ESI-RR@10 Diversity EILD-R@10 EILD-RR@10

58. Datasets (1/2) 58 G1 dataset ● Provided by Globo.com, the most popular news portal in Brazil ● Made publicly available by this research for the community https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom

59. 59 G1 (globo.com) Adressa Language Portuguese Norwegian Period (days) 16 16 # users 322 k 314 k # sessions 1,048 k 982 k # clicks 2,988 k 2,648 k # articles 46 k 13 k Avg. session length * 2.84 2.70 * After removing single-click sessions and sessions with more than 20 clicks. Datasets (2/2)

60. Baseline algorithms 60 Neural methods GRU4Rec A landmark neural architecture using RNNs for session-based recommendation SR-GNN A recently state-of-the-art architecture for session-based recommendation based on GNN. Frequent patterns methods CO A simple method of co-occurrence based on association rules of length two. SR Sequential rules of size two. kNN methods Item-kNN Most similar articles to the last read one, in terms of the cosine similarity between the vector of their occurrence in sessions. V-SkNN Compares the entire active session with past (neighboring) sessions to determine items to be recommended. The similarity function emphasizes items that appear later within the session. Non-personalized methods RP Recently Popular - Most viewed articles during the last hour. CB Content-Based - Most similar articles (content embedding) to the last user click.

61. Main Results

62. Research Questions How does the proposed hybrid RNN-based architecture – the CHAMELEON – perform in the news domain, in terms of recommendation quality factors (accuracy, item coverage, novelty, and diversity), compared to existing approaches for session-based recommendation? 62 DLRS @ RecSys, 2018 IEEE Access, 2019 (Qualis A2, Eng. IV) 1

63. G1 dataset +14.2% +19.6% News recommendation quality of session-based algorithms 1 63 Other neural-based methods

64. +14.2% +19.6% Adressa dataset +11.6% +13.2% News recommendation quality of session-based algorithms 1

65. 65 Average MRR@10 by hour (evaluation each 5 hours), during 16 days G1 dataset News recommendation quality of session-based algorithms 1

66. 66 Average MRR@10 by hour (evaluation each 5 hours), during 16 days Adressa dataset News recommendation quality of session-based algorithms 1

67. 67 Recommendation Accuracy (HR@10 ) of algorithms (lines) x Avg. Normalized Popularity (bars) by session click order Adressa dataset News recommendation quality of session-based algorithms 1 G1 dataset

68. News recommendation quality of session-based algorithms 68 Evaluation of other quality factors G1 dataset 1

69. News recommendation quality of session-based algorithms 69 Evaluation of other quality factors 1Adressa dataset

70. Research Questions 70 What is the effect on news recommendation quality factors of leveraging different types of information in a neural-based hybrid recommender system? IEEE Access, 2019 (Qualis A2, Eng. IV) 2

71. 71 The eﬀect of diﬀerent types of information 2 Input Configuration Feature Sets IC1 Article Id IC2 IC1 + Article Context (Novelty and Recency) IC3 IC2 + the Article Content Embeddings (ACE) learned by the supervised instantiation of the ACR module IC4 IC3 + Article Metadata IC5 IC4 + User Context

72. 72 The eﬀect of diﬀerent types of information 2

73. Research Questions 73 What is the effect on news recommendation quality of using different textual representations, produced by statistical NLP and Deep NLP techniques? INRA @ RecSys, 2019 3

74. 74 The eﬀect of diﬀerent content representations 3ACRmodule instantiations

75. 75 Recommender HR@10 MRR@10 No-ACE 0.6281 0.3066 Supervised CNN 0.6585 0.3395 GRU 0.6585 0.3388 Unsupervised SDA-GRU 0.6418 0.3160 W2V*TF-IDF 0.6575 0.3291 LSA 0.6686* 0.3423 doc2vec 0.6368 0.3119 Comparison of different content encodings for CHAMELEON The eﬀect of diﬀerent content representations G1 dataset 3

76. 76Comparison of different content encodings for CHAMELEON The eﬀect of diﬀerent content representations Adressa dataset 3 Recommender HR@10 MRR@10 No-ACE 0.6816 0.3252 Supervised CNN 0.6860 0.3333 GRU 0.6856 0.3327 Unsupervised SDA-GRU 0.6905 0.3360 W2V*TF-IDF 0.6913 0.3402 LSA 0.6935 0.3403 doc2vec 0.6898 0.3402

77. Research Questions Is the proposed hybrid RNN-based architecture able to reduce the problem of item cold-start in the news domain, compared to other existing approaches for session-based recommendation? 77 4

78. 78 The eﬀect on item cold-start 4 # Batches before First Recommendation (BFR@n) G1 dataset

79. 79 The eﬀect on item cold-start 4 Adressa dataset # Batches before First Recommendation (BFR@n)

80. Research Questions Is the proposed approach effective to balance the competing quality factors of accuracy and novelty? 80 IEEE Access, 2019 (Qualis A2, Eng. IV) 5

81. 81 Balancing the trade-oﬀ between Accuracy and Novelty 5 G1 dataset Adressa dataset

82. Conclusion

83. Main Contributions 83 1. The investigation of the stated research questions related to the effectiveness of session-based recommendation algorithms on the news domain 2. The design, implementation, and evaluation of a Deep Learning meta-architecture – the CHAMELEON – for hybrid and contextual session-based news recommendation

84. Complementary Contributions 84 1. The elaboration of a Conceptual Model of factors that affect news relevance 2. The four instantiations of ACR and NAR modules of CHAMELEON 3. The open-source implementation of CHAMELEON 1 4. The temporal offline evaluation protocol for news recommendation 5. The adaptation of rank- and relevance-sensitive novelty and diversity metrics Additional Contribution 6. The preprocessing and sharing of two novel datasets for evaluation of hybrid and contextual RS for news (G1) 2 and articles (CI&T Desktop) 3 1 https://github.com/gabrielspmoreira/chameleon_recsys 2 https://www.kaggle.com/gspmoreira/news-portal-user-interactions-by-globocom 3 https://www.kaggle.com/gspmoreira/articles-sharing-reading-from-cit-deskdrop

85. Recommendations 85 1. Online evaluation of CHAMELEON in a live news portal 2. To balance more than two quality factors (including diversity) 3. To adapt CHAMELEON for session-aware news recommendation 4. To explore different negative sample approaches (e.g., by geographic region) 5. Explore other Deep NLP techniques (e.g. BERT) to produce better article content embeddings 6. To deepen the investigation of the RQ4 about the item cold-start problem (next paper)

86. Suggestions for Future Works 86 1. Instantiate the SER sub-module with a neural network architecture other than RNNs (e.g., CNN, QRNN, GNN) 2. Investigate attention mechanisms as a way to improve accuracy and provide recommendation explanations 3. Investigate “noisy outliers” in the user profiles 4. Adapt CHAMELEON for contextual and hybrid session-based recommendations in other domains like e-commerce, entertainment and media

87. Published Articles 87 1. MOREIRA, Gabriel de Souza Pereira; SOUZA JUNIOR, G. J. A. ; CUNHA, A. M. . Comparing ofﬂine and online recommender system evaluations on long-tail distributions. In: ACM Recommender Systems Conference, 2015, Viena, Austria. Poster Proceedings of ACM RecSys 2015, 2015. v. 1441. p. 2. MOREIRA, GABRIEL DE SOUZA P.; SOUZA, GILMAR . A Recommender System to tackle Enterprise Collaboration. In: the 10th ACM Conference, 2016, Boston. Proceedings of the 10th ACM Conference on Recommender Systems - RecSys '16 (Demo paper). New York: ACM Press, 2016. p. 395. DOI https://doi.org/10.1145/2959100.2959115 3. MOREIRA, GABRIEL. CHAMELEON: a Deep Learning Meta-Architecture for News Recommender Systems. In: 12th ACM Conference on Recommender Systems - RecSys'18, 2018, Vancouver. Proceedings of the 12th ACM Conference on Recommender Systems RecSys'18 - Doctoral Symposium. New York: ACM Press, 2018. DOI https://doi.org/10.1145/3240323.3240331 4. MOREIRA, GABRIEL; FERREIRA, FELIPE ; DA CUNHA, ADILSON MARQUES . News Session-Based Recommendations using Deep Neural Networks. Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems - DLRS 2018. New York: ACM Press, 2018. p.15. DOI https://doi.org/10.1145/3270323.3270328 5. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES . On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems. Proceedings of the 7th Workshop on News Recommendation and Analytics (INRA 2019), in conjunction with RecSys 2019, September 19, 2019, Copenhagen 6. MOREIRA, GABRIEL; JANNACH, DIETMAR ; DA CUNHA, ADILSON MARQUES .. "Contextual Hybrid Session-Based News Recommendation With Recurrent Neural Networks." IEEE Access 7 (2019): 169185-169203.

88. CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems Gabriel de Souza Pereira Moreira Advisor: Prof. Dr. Adilson Marques da Cunha Doctoral Thesis Defense Instituto Tecnológico de Aeronáutica 09/12/2019

[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

You are reading a preview.

Check these out next

[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems

More Related Content

Slideshows for you (20)

Similar to [Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems (20)

More from Gabriel Moreira (19)

Recently uploaded (20)

[Phd Thesis Defense] CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems