Speaker: Iván Palomares Carrascosa.
DaSCI Andalusian Institute of Data Science and Artificial Intelligence.
Key words: recommender systems, information retrieval, reciprocal recommendation, people recommendation, online dating, recruitment, online learning, social networks, socia media, data fusion, preference aggregation.
出典:Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko
Facebook AI
公開URL : https://arxiv.org/abs/2005.12872
概要:Detection Transformer(DETRという)という新しいフレームワークによって,non-maximum-supressionやアンカー生成のような人手で設計する必要なく、End-to-Endで画像からぶった検出を行う手法を提案しています。物体検出を直接集合予測問題として解くためのtransformerアーキテクチャとハンガリアン法を用いて二部マッチングを行い正解と予測の組み合わせを探索しています。Attentionを物体検出に応用しただけでなく、競合手法であるFaster R-CNNと同等の精度を達成しています。
出典:Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko
Facebook AI
公開URL : https://arxiv.org/abs/2005.12872
概要:Detection Transformer(DETRという)という新しいフレームワークによって,non-maximum-supressionやアンカー生成のような人手で設計する必要なく、End-to-Endで画像からぶった検出を行う手法を提案しています。物体検出を直接集合予測問題として解くためのtransformerアーキテクチャとハンガリアン法を用いて二部マッチングを行い正解と予測の組み合わせを探索しています。Attentionを物体検出に応用しただけでなく、競合手法であるFaster R-CNNと同等の精度を達成しています。
- The document discusses linear regression models and methods for estimating coefficients, including ordinary least squares and regularization methods like ridge regression and lasso regression.
- It explains how lasso regression, unlike ordinary least squares and ridge regression, has the property of driving some of the coefficient estimates exactly to zero, allowing for variable selection.
- An example using crime rate data shows how lasso regression can select a more parsimonious model than other methods by setting some coefficients to zero.
1) Canonical correlation analysis (CCA) is a statistical method that analyzes the correlation relationship between two sets of multidimensional variables.
2) CCA finds linear transformations of the two sets of variables so that their correlation is maximized. This can be formulated as a generalized eigenvalue problem.
3) The number of dimensions of the transformed variables is determined using Bartlett's test, which tests the eigenvalues against a chi-squared distribution.
Study of Recommendation System Used In Tourism and Travelijtsrd
This study is based on Recommendation Systems and its Types used in Tourism and Travel Website. Recommendation Systems are used in websites so that it can recommend item to a user based on his her interest and on the basis of user profile. In this paper, I design a recommender system for recommending tourist places based on content based and collaborative filtering techniques. This method combines both behavioural and content aspects of recommendations. The flow for the research is that first of all using cosine similarity, weighted ratings and Location APIs we build a content based system. The process is carried out by comparing the features of the item with respect to the user’s preferences. Then followed by collaborative filtering techniques such as Correlation and K Nearest neighbour in which items predict the interest of the user on an activity considering the evaluation that a particular user has given to similar activities. Shikhar "Study of Recommendation System Used In Tourism and Travel" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-6 | Issue-1 , December 2021, URL: https://www.ijtsrd.com/papers/ijtsrd47922.pdf Paper URL: https://www.ijtsrd.com/computer-science/other/47922/study-of-recommendation-system-used-in-tourism-and-travel/shikhar
This document summarizes mathematical methods of tensor factorization applied to recommender systems. It discusses motivations and contributions, information overload and recommender systems, matrix and tensor factorization techniques in recommender system literature such as matrix factorization, singular value decomposition, high-order singular value decomposition, and parallel factor analysis. It also covers challenges in context-aware recommender systems and proposed solutions to incorporate contextual information.
- The document discusses linear regression models and methods for estimating coefficients, including ordinary least squares and regularization methods like ridge regression and lasso regression.
- It explains how lasso regression, unlike ordinary least squares and ridge regression, has the property of driving some of the coefficient estimates exactly to zero, allowing for variable selection.
- An example using crime rate data shows how lasso regression can select a more parsimonious model than other methods by setting some coefficients to zero.
1) Canonical correlation analysis (CCA) is a statistical method that analyzes the correlation relationship between two sets of multidimensional variables.
2) CCA finds linear transformations of the two sets of variables so that their correlation is maximized. This can be formulated as a generalized eigenvalue problem.
3) The number of dimensions of the transformed variables is determined using Bartlett's test, which tests the eigenvalues against a chi-squared distribution.
Study of Recommendation System Used In Tourism and Travelijtsrd
This study is based on Recommendation Systems and its Types used in Tourism and Travel Website. Recommendation Systems are used in websites so that it can recommend item to a user based on his her interest and on the basis of user profile. In this paper, I design a recommender system for recommending tourist places based on content based and collaborative filtering techniques. This method combines both behavioural and content aspects of recommendations. The flow for the research is that first of all using cosine similarity, weighted ratings and Location APIs we build a content based system. The process is carried out by comparing the features of the item with respect to the user’s preferences. Then followed by collaborative filtering techniques such as Correlation and K Nearest neighbour in which items predict the interest of the user on an activity considering the evaluation that a particular user has given to similar activities. Shikhar "Study of Recommendation System Used In Tourism and Travel" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-6 | Issue-1 , December 2021, URL: https://www.ijtsrd.com/papers/ijtsrd47922.pdf Paper URL: https://www.ijtsrd.com/computer-science/other/47922/study-of-recommendation-system-used-in-tourism-and-travel/shikhar
This document summarizes mathematical methods of tensor factorization applied to recommender systems. It discusses motivations and contributions, information overload and recommender systems, matrix and tensor factorization techniques in recommender system literature such as matrix factorization, singular value decomposition, high-order singular value decomposition, and parallel factor analysis. It also covers challenges in context-aware recommender systems and proposed solutions to incorporate contextual information.
This document provides an introduction to recommendation systems. It discusses three main recommendation approaches: content-based, collaborative filtering, and hybrid methods. It also outlines Netflix's $1 million Netflix Prize competition to improve their recommendation algorithm. Key points covered include defining recommendation systems, different recommendation types, collecting and extrapolating user ratings data, limitations of different approaches, and evaluating recommendation quality.
Beyond Collaborative Filtering: Learning to Rank Research ArticlesMaya Hristakeva
This document provides an overview of Elsevier's work using machine learning techniques like collaborative filtering and learning to rank to improve article recommendations for researchers. It discusses using collaborative filtering on browsing logs to initially recommend related articles, then using learning to rank to re-rank those recommendations higher based on features like reputation, topics, citations and user engagement data. Evaluation shows the learning to rank approach improved user engagement by 9-10% over collaborative filtering alone. Future work may explore alternative approaches like graph-based methods or deep learning.
The document summarizes a research paper that proposes a personalized recommendation approach combining social network factors like interpersonal interest similarity and interpersonal rating behavior similarity. It uses probabilistic matrix factorization to predict ratings by considering these social network factors. The approach is evaluated on two large real-world social rating datasets and shows improved performance over approaches that only use social network information.
Random Walk by User Trust and Temporal Issues toward Sparsity Problem in Soci...Sc Huang
Random Walk by User Trust and Temporal Issues toward Sparsity Problem in Social Tagging Recommender Systems
The document discusses using random walks and temporal factors to address sparsity problems in social tagging recommender systems. It introduces related work on item-based collaborative filtering, random walk recommendations, and models that learn influence probabilities. It then describes using random walks starting from users or items, and incorporating trust networks and influence powers to provide recommendations. Finally, it discusses addressing cold start problems, temporal decay issues, and experiment design.
IRJET- A Survey on Recommender Systems used for User Service Rating in Social...IRJET Journal
This document summarizes research on recommender systems used for user service ratings in social networks. It first discusses how recommender systems predict user ratings using collaborative, content-based, and hybrid filtering techniques. It then reviews related work on collaborative, content-based, and hybrid recommendation approaches. Challenges like cold starts are also discussed. The document concludes that combining personal interests, social similarities and influences into a unified framework can improve rating predictions.
Recommender systems give suggestion according
to the user preferences. The number of contents and books in a
university size library is enormous and a better than ever.
Readers find it extremely difficult to locate their favorite books.
Even though they could possibly find best preferred book by
the user, finding another similar book to the first preferred
book seems as if finding an in nail the ocean. That is because
the second preferred book might be at very last edge of long
tail. So recommender system is often a requirement in library
that should be considers and need it to come into make the
above finding similar. They have become fundamental
applications in electronic commerce and information retrieval,
providing suggestions that effectively crop large information
spaces so that users are directed toward those items that best
meet their needs and preferences. A variety of techniques have
been suggested for performing recommendation, including
collaborative technique and its three methods which are Slope
One used for rating prediction, Pearson’s correlation used for
finding the similarity between users and last but not the least
item to item similarity. To upgrade the performance, these
methods have sometimes been combined in hybrid
recommendation technique.
This document summarizes several research papers on improving recommendation systems using item-based collaborative filtering approaches. It discusses challenges with traditional collaborative filtering like data sparsity and scalability. It then summarizes various item-based recommendation algorithms that analyze item relationships to indirectly compute recommendations. These include item-item similarity techniques like cosine similarity and adjusting for average ratings. The document also reviews literature on combining item categories and interestingness measures to improve accuracy. Overall, it analyzes different item-based collaborative filtering techniques to address challenges and provide high quality, personalized recommendations at scale.
Building a Recommender systems by Vivek Murugesan - Technical Architect at Cr...Rajasekar Nonburaj
The topic presented at the "Datascience Chennai June Meetup"
"Building a Recommender systems" by Vivek Murugesan - Technical Architect at Crayon Data. Check more at https://www.meetup.com/datasciencechn
Recommender systems attempt to predict a user's preferences to recommend relevant items. They use algorithms like collaborative filtering on user data to deliver personalized recommendations. To scale, techniques like clustering users and dimensionality reduction are used. Learning to rank models combine user preferences, item attributes, and business goals to generate a ranked list of recommendations. Open source datasets and frameworks can help get started with implementing recommender systems.
Social Recommender Systems Tutorial - WWW 2011idoguy
The document discusses social recommender systems and various approaches used in them. It covers fundamental recommendation techniques like collaborative filtering, content-based recommendation, and knowledge-based recommendation. It also discusses using tags, social relationships, and temporal data in recommendations. Evaluation of recommender systems and challenges are also summarized.
Harvesting Intelligence from User Interactions R A Akerkar
This document discusses how to harvest intelligence from user interactions on websites. It explains that as users share opinions, content, and participate in online communities, data is generated that can be converted into intelligence to personalize websites. It describes how collecting diverse opinions from many users can lead to "wise crowds" and collective intelligence. The key is to allow user interactions, learn about users in aggregate, and personalize content using this data. Content and collaborative filtering are approaches to build user and item profiles to detect meaningful relationships and make recommendations. The goal is to transform applications from being content-centric to being user-centric using collective intelligence.
Collaborative filtering is a technique used in recommender systems to predict a user's preferences based on other similar users' preferences. It involves collecting ratings data from users, calculating similarities between users or items, and making recommendations. Common approaches include user-user collaborative filtering, item-item collaborative filtering, and probabilistic matrix factorization. Recommender systems are evaluated both offline using metrics like MAE and RMSE, and through online user testing.
This document provides an introduction to recommender systems. It discusses how recommender systems help users discover new content in the "age of recommendation" by providing personalized recommendations. Common techniques for building recommender systems include collaborative filtering, which looks at ratings from similar users to provide recommendations. Memory-based collaborative filtering uses item or user similarities to make predictions, while model-based approaches like matrix factorization use dimensionality reduction techniques to learn latent factors from user-item rating matrices. Matrix factorization approaches like SVD have been shown to provide accurate recommendations.
Alluding Communities in Social Networking Websites using Enhanced Quasi-cliqu...IJMTST Journal
1) The document proposes an enhanced technique to recommend communities to users in social networks based on the user's interests and their strong friends.
2) It identifies a user's area of interest by analyzing their posts and classifying keywords. It then determines the user's strong friends based on an enhanced quasi-clique technique, considering interaction strength.
3) Communities are recommended by considering both the user's interests and strong friends. This provides a more precise recommendation than only considering strong friends.
Recommender systems aim to recommend items like books, movies, or products to users based on their preferences. There are two main approaches: collaborative filtering, which recommends items liked by similar users, and content-based filtering, which recommends items similar to those a user has liked based on item attributes. Both have strengths and weaknesses, so hybrid systems combining the approaches can provide the best recommendations.
The document describes a proposed approach for inferring implicit topical interests of users on Twitter. It discusses related work on detecting user interests from social media using bag-of-words, topic modeling, and bag-of-concepts approaches. The proposed approach models user interests as a graph-based link prediction problem over a heterogeneous graph incorporating user followerships, explicit interests, and topic relatedness. It evaluates different variants of the model and finds semantic relatedness of topics to be most effective for identifying implicit user interests.
Extracting, Mining and Predicting Users’ Interests from Social MediaFattane Zarrinkalam
1) Implicit user interest modeling learns users' potential interests that were not explicitly mentioned by analyzing relationships between users and topics of interest. This approach is important for passive users who do not generate much content.
2) Relationships between users, such as followers on Twitter, and relationships between topics, such as hierarchies in Wikipedia, can provide clues about a user's implicit interests. A user's interests may be semantically related or similar to the interests of other users they follow or topics that are related in a knowledge base.
3) Techniques like collaborative filtering and frequent pattern mining can identify implicit interests by measuring relatedness between a user's explicit interests and other users' interests or topics in a collection.
IRJET- Review on Different Recommendation Techniques for GRS in Online Social...IRJET Journal
This document reviews different recommendation techniques for group recommender systems (GRS) in online social networks. It discusses traditional recommender approaches like content-based filtering and collaborative filtering. It also reviews related work applying opinion dynamics models and weight matrices to GRS. The document concludes that using a smart weights matrix to consider relationships between group members' preferences in a recommendation process improves aggregation and ensures consensus, providing the best way to recommend items to a complete group.
Similar to ACM SIGIR 2020 Tutorial - Reciprocal Recommendation: matching users with the right users (20)
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
Introduction to Jio Cinema**:
- Brief overview of Jio Cinema as a streaming platform.
- Its significance in the Indian market.
- Introduction to retention and engagement strategies in the streaming industry.
2. **Understanding Retention and Engagement**:
- Define retention and engagement in the context of streaming platforms.
- Importance of retaining users in a competitive market.
- Key metrics used to measure retention and engagement.
3. **Jio Cinema's Content Strategy**:
- Analysis of the content library offered by Jio Cinema.
- Focus on exclusive content, originals, and partnerships.
- Catering to diverse audience preferences (regional, genre-specific, etc.).
- User-generated content and interactive features.
4. **Personalization and Recommendation Algorithms**:
- How Jio Cinema leverages user data for personalized recommendations.
- Algorithmic strategies for suggesting content based on user preferences, viewing history, and behavior.
- Dynamic content curation to keep users engaged.
5. **User Experience and Interface Design**:
- Evaluation of Jio Cinema's user interface (UI) and user experience (UX).
- Accessibility features and device compatibility.
- Seamless navigation and search functionality.
- Integration with other Jio services.
6. **Community Building and Social Features**:
- Strategies for fostering a sense of community among users.
- User reviews, ratings, and comments.
- Social sharing and engagement features.
- Interactive events and campaigns.
7. **Retention through Loyalty Programs and Incentives**:
- Overview of loyalty programs and rewards offered by Jio Cinema.
- Subscription plans and benefits.
- Promotional offers, discounts, and partnerships.
- Gamification elements to encourage continued usage.
8. **Customer Support and Feedback Mechanisms**:
- Analysis of Jio Cinema's customer support infrastructure.
- Channels for user feedback and suggestions.
- Handling of user complaints and queries.
- Continuous improvement based on user feedback.
9. **Multichannel Engagement Strategies**:
- Utilization of multiple channels for user engagement (email, push notifications, SMS, etc.).
- Targeted marketing campaigns and promotions.
- Cross-promotion with other Jio services and partnerships.
- Integration with social media platforms.
10. **Data Analytics and Iterative Improvement**:
- Role of data analytics in understanding user behavior and preferences.
- A/B testing and experimentation to optimize engagement strategies.
- Iterative improvement based on data-driven insights.
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
Open Source Contributions to Postgres: The Basics POSETTE 2024ElizabethGarrettChri
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
ACM SIGIR 2020 Tutorial - Reciprocal Recommendation: matching users with the right users
1. ACM SIGIR’20 Tutorial Iván Palomares Carrascosa ivanpc@ugr.es
Reciprocal Recommendation
Matching Users with the Right Users
2. 26 July 2020, ACM SIGIR’20, Virtual Event
About the presenter
Iván Palomares Carrascosa
DaSCI Research Institute, Granada, Spain
Research interests:
- Recommender Systems
- Group and multi-criteria decision-making
- Preference modelling and fusion
- Applications: participatory decisions, group
recommendation, social recommendation, tourism,
health.
Website: www.ivanpc.com
3. 26 July 2020, ACM SIGIR’20, Virtual Event
Tutorial Materials
Decision Support and Recommender Systems Lab website:
http://dsrs-lab.ugr.es
Go to ‘Events’ à ‘ACM SIGIR’20 Tutorial’
- Slides (PDF)
- Paper: Reciprocal Recommender Systems: Analysis of State-of-the-Art Literature,
Challenges and Opportunities on Social Recommendation.
4. 26 July 2020, ACM SIGIR’20, Virtual Event
Contents
PART I
• Reciprocal Recommender Systems (RRS)
• The Reciprocal Recommendation process I: prediction
• The Reciprocal Recommendation process II: fusion
Virtual coffee break
PART II
• Applications of RRS
• Evaluating RRS
• Challenges and future directions
• Discussion activity
5. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
Recommender Systems (RS)
• Provide users with items (products, contents, …) they might be
interested in, by analyzing their preferences, needs or behavior.
• Solution for the information overload problem in Internet sites.
• Applications: e-commerce, entertainment, retail, tourism, …
What does a RS do?
• Predict user-item preferences: how much a user may like an
unseen item by him/her.
• Recommend the items predicted as most liked by the user.
• As the user interacts with the system, the system builds insight of
the user’s preferences.
6. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
Formal RS definition: Given a user x∈U, a recommender R(x) is a
system that recommends a list of items R ⊂I such that the degree of
preference px,i by x towards every item i∈ R is stronger than the
preference degree by x towards any item i’∉ R:
7. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
Reciprocal Recommender Systems
• Recommend users to each other.
• In their simplest form, they suggest pairs of users to connect.
• Both users must reciprocate – both of them should be satisfied with
the ”matching” suggestion.
• Bidirectional preference à determine mutual compatibility
between users, e.g. reciprocal preference score 𝑝%↔'
8. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
From traditional item-to-user to reciprocal recommendation
Item-to-user recommender Reciprocal Recommender
Recommendations for 𝑥 ∈ 𝑈 are items i ∈ 𝐼 Recommendations for 𝑥 ∈ 𝑈 are users 𝑦 ∈ 𝑈
Items can be usually recommended to
multiple users
In some applications, if both x and y accept the
matching recommendation, then they are no longer
available for being recommended to anyone else.
A successful recommendation does not
imply that the user leaves the system
In some applications, users may not need using the
system and leave it after a successful
recommendation
Success is determined by the user receiving
the recommendation
Both x and y must be satisfied with the
recommendation to deem it as successful
9. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
Single-class vs two-class RRS
Single-class RRS
- Homogeneous set of users U
- Any two users in U can be recommended to each other
- Applications: symmetric social networks, matching learners, homosexual dating, shared-
economy, …
Two-class RRS
- U subdivided in two disjoint subsets of users or classes, e.g. U = M ∪ F
- Given x ∈ 𝑀, only users in the other class can be recommended, y ∈ 𝐹
- Applications: heterosexual dating, recruitment, student-mentor matching, …
10. 26 July 2020, ACM SIGIR’20, Virtual Event
Reciprocal Recommender Systems
(RRS)
General RRS conceptual model
12. 26 July 2020, ACM SIGIR’20, Virtual Event
Content-based Filtering (CB)
Rely on content features or item metadata to recommend similar
content to what I liked.
How do CB principles apply in reciprocal recommendation?
5/5
2/5
Recommendation
13. 26 July 2020, ACM SIGIR’20, Virtual Event
CB in Reciprocal Recommendation
• Users typically have profiles with information about themselves.
• They may also have information about types of users they are
interested in.
• Explicit preferences: attributes of users sought, ratings, …
• Implicit preferences: they are learnt from user activity in the
system: EoIs, viewed users’ profiles, messaging, etc.
Given a subject user x, in a CB-RRS, knowledge is built about the
type/attributes of users liked by x, i.e. her preferences.
Then a user y whose profile aligns with x’s preferences, is likely to be
recommended to x …. (or is he?)
14. 26 July 2020, ACM SIGIR’20, Virtual Event
CB in Reciprocal Recommendation
• Users typically have profiles with information about themselves.
• They may also have information about type of users they are
interested in.
• Explicit preferences: attributes of users sought, ratings, …
• Implicit preferences: they are learnt from user activity in the
system: EoIs, viewed users’ profiles, messaging, etc.
Given a subject user x, in a CB-RRS, knowledge is built about the
type/attributes of items (other users) liked by x, i.e. her preferences.
A user y whose profile matches x’s preferences, is likely to be
recommended to x iff x’s profile also matches y’s preferences.
Reciprocity!
15. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
Basic Notation
• A: Set of profile attributes: gender, age, height, eye color, …
• x: the user for whom recommendations will be produced.
Subject user = recommendee = active user = target user
• Ux: Profile of user x
𝑈% = {𝑣%,4: ∀𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒𝑠 𝑎 ∈ 𝐴}
• vx,a: Value of attribute 𝑎 ∈ 𝐴 for user x.
L. Pizzato, Pizzato, L., Rej, T., Chung, T., Koprinska, I., & Kay, J. (2010). RECON: A
reciprocal recommender for online dating. Proceedings ACM RecSys ‘10, pp. 207-214.
RECON utilizes profile information and preferences inferred from
messaging behavior to make reciprocal recommendations.
16. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
Basic Notation
• Mx,* : Users x sent an EoI to (e.g. a message).
• M+
x,* : Users x sent an EoI to and responded positively
(reciprocated).
• M-
x,* : Users x sent an EoI to and responded negatively (rejected).
• M*,x : Users who sent an EoI to x.
• M+
*,x : Users who sent an EoI to x and x responded positively
(reciprocated).
• M-
*,x : Users x sent an EoI to and x responded negatively
(rejected).
• Rx : List of recommended users y for x. It holds 𝑅% ∩ 𝑀%,∗ ∪ 𝑀∗,% = ∅
17. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
RECON key ideas (I) – building a preference model for x
• For every x and profile attribute 𝑎 ∈ 𝐴, a frequency distribution is
built, representing the preferences Px of x on values of that attribute.
• This is done by looking at occurrences of attributes’ values in (1)
users contacted by x, and (2) users to whom x replied positively.
Mx,*
M+
*,x
a = body shape
18. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
RECON key ideas (I) – building a preference model for x
• Frequency distributions are built for discrete attributes.
• But, how about continuous attributes, e.g. height and weight?
• Not considered in the original model, but the average value exhibited by all users
in 𝑀%,∗ ∪ 𝑀∗,%
E could be taken.
Images source: Pizzato et al. (2010)
19. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
RECON key ideas (II) – calculate compatibility with unknown users
• Uy: User y profile, such that 𝑈' = {𝑣',4: ∀𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒𝑠 𝑎 ∈ 𝐴}
• For discrete attributes, add the frequencies in Px associated to vy,a
Example with four discrete attributes:
User y is 1.70cm tall, slim, social,
single and has high-school education.
s = 0.25 + 0.4 + 1 + 0.15 = 1.8 (out of 4)
Then the compatibility or preference by
x towards y is px,y = 1.8/4 = 0.45
Images source: Pizzato et al. (2010)
20. 26 July 2020, ACM SIGIR’20, Virtual Event
RECON: A content-based RRS
RECON key ideas (II) – calculate compatibility with unknown users
• The px,y values like the one we just calculated would be enough to
produce a top-N recommendation list … if we only needed to consider
user x’s preferences.
• But we also need to account for y’s preferences à Reciprocity!
Let’s go the other way round:
• Assume that for Py and Ux we now get py,x = 0.6.
• We then aggregate both preference scores to get the reciprocal
preference score of mutual preference between x and y.
Harmonic mean IN ACTION: 𝒑 𝒙↔𝒚 =
𝟐
𝟎.𝟒𝟓N𝟏E𝟎.𝟔N𝟏 = 0.514
21. 26 July 2020, ACM SIGIR’20, Virtual Event
Collaborative Filtering (CF)
Recommend items liked by similar users to me.
How do CF principles apply in reciprocal recommendation?
Recommendation
Liked Liked
22. 26 July 2020, ACM SIGIR’20, Virtual Event
CF in Reciprocal Recommendation
• CB-RRS accounted for similarity between the subject users’
preferences and unknown object users (treated as “items”).
• In CF-RRS we observe the preferences of users who interact
similarly (they liked similar object users) as the target user.
• User profile information is less relevant in CF, now we concentrate
on analyzing user-user activity patterns, and identifying users
with similar activity to us à neighbor users.
Given a subject user x, a CF-RRS analyses users’ interactions with
other users to identify users with similar interactions to x.
23. 26 July 2020, ACM SIGIR’20, Virtual Event
CF in Reciprocal Recommendation
User taste and user attractiveness
(1) Taste – determined by active user interaction:
• If two users x,z in the same class initiate positive interactions with
several users y1, y2, yn in common, x and z have similar taste.
24. 26 July 2020, ACM SIGIR’20, Virtual Event
CF in Reciprocal Recommendation
User taste and user attractiveness
(2) Attractiveness – determined by passive user interaction:
• If two users x,z in the same class receive positive interactions from
various other users y1, y2, yn in common, x and z have similar
attractiveness.
25. 26 July 2020, ACM SIGIR’20, Virtual Event
CF in Reciprocal Recommendation
CF assumptions in Reciprocal Recommendation
y should be recommended to x when: y likes people with similar
attractiveness to x and x likes people with similar attractiveness to y,
or equivalently
when people with similar taste to y like x and people with similar
taste to x like y.
If people with similar taste to x like y, x will like y.
If people with similar taste to y like x, y will like x.
If x likes people with similar attractiveness to y, x will like y.
If y likes people with similar attractiveness to x, y will like x.
X. Cai et al. (2010). Collaborative Filtering for People to People
Recommendation in Social Networks. Proceedings AI 2010, pp. 476-485.
26. 26 July 2020, ACM SIGIR’20, Virtual Event
RCF: A collaborative filtering RRS
Basic Ideas
• Nearest-neighbour strategy
• The preference px,y is determined based on the similarity between the
[taste / attractiveness] of x and users z who positively interacted with y.
• Interest similarity: Attractiveness similarity:
P. Xia, B. Liu, Y. Sun, C. Chen. (2015). Reciprocal recommendation system for online dating.
Proceedings ASONAM’15, pp. 234-241.
RCF is a family of algorithms, some of them memory-based CF,
relying on different neighborhood and similarity functions.
27. 26 July 2020, ACM SIGIR’20, Virtual Event
RCF: A collaborative filtering RRS
• Interest similarity: Attractiveness similarity:
• Neighborhood function: Users who had some interaction with x (in the
opposite class to x in a two-class RRS modelled as a bipartite graph).
𝑁𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑥 = 𝐸𝑜𝐼VWXY 𝑥 𝑁𝑒𝑖𝑔ℎ𝑏𝑜𝑟 𝑥 = 𝐸𝑜𝐼ZX(𝑥)
• Different combinations of similarity measure + neighbourhood
function give rise to different “versions” of RCF.
28. 26 July 2020, ACM SIGIR’20, Virtual Event
RCF: A collaborative filtering RRS
General RCF procedure
29. 26 July 2020, ACM SIGIR’20, Virtual Event
RCF: A collaborative filtering RRS
General RCF procedure For all users 𝑧 ∈ 𝑁𝑒𝑖𝑔ℎ𝑏𝑜𝑟^ 𝑦 ,
calculate px,y as the average of the
similarities sim(x,z)
For all users 𝑣 ∈ 𝑁𝑒𝑖𝑔ℎ𝑏𝑜𝑟_ 𝑥 ,
calculate py,x as the average of the
similarities sim(y,v)
30. 26 July 2020, ACM SIGIR’20, Virtual Event
RCF: A collaborative filtering RRS
General RCF procedure
• Preference fusion: Reciprocal preference 𝑝%↔` calculated using the
harmonic mean.
• Filter for x the N users y with highest 𝑝%↔`
Results:
• RCF more effective than prior CB-RRS in precision and recall.
• Relatively simple to implement and understand.
• Memory-based: no pre-trained model required.
• Drawback: Computational/temporal cost on large datasets.
31. 26 July 2020, ACM SIGIR’20, Virtual Event
LFRR: Latent Factor Model RRS
Basic Ideas
• Two preference matrices: female-to-male preferences and male-to-
female preferences.
• Two LF models (one per matrix) trained via matrix factorization.
J. Neve, I. Palomares (2019). Latent Factor Models and Aggregation Operators for Collaborative
Filtering in Reciprocal Recommender Systems (Long Paper). Proc. ACM Recsys’19, pp.219-227.
LFRR is a model-based CF-RRS that determines latent user attributes.
32. 26 July 2020, ACM SIGIR’20, Virtual Event
LFRR: Latent Factor Model RRS
Basic Ideas
• Two preference matrices: female-to-male preferences and male-to-
female preferences.
• Two LF models (one per matrix) trained via matrix factorization.
J. Neve, I. Palomares (2019). Latent Factor Models and Aggregation Operators for Collaborative
Filtering in Reciprocal Recommender Systems (Long Paper). Proc. ACM Recsys’19, pp.219-227.
LFRR is a model-based CF-RRS that determines latent user attributes.
dot
product of two
latent vectors
Known
ratings
33. 26 July 2020, ACM SIGIR’20, Virtual Event
LFRR: Latent Factor Model RRS
Basic Ideas
• Two preference matrices: female-to-male preferences and male-to-
female preferences.
• Two LF models (one per matrix) trained via matrix factorization.
J. Neve, I. Palomares (2019). Latent Factor Models and Aggregation Operators for Collaborative
Filtering in Reciprocal Recommender Systems (Long Paper). Proc. ACM Recsys’19, pp.219-227.
LFRR is a model-based CF-RRS that determines latent user attributes.
Stochastic Gradient
Descent (SGD) Row: user
preferences
Column: user
traits
34. 26 July 2020, ACM SIGIR’20, Virtual Event
LFRR: Latent Factor Model RRS
Preference px,y estimated as the resulting dot product of latent
vectors, 𝒑% ⋅ 𝒒'
c
, after applying SGD.
35. 26 July 2020, ACM SIGIR’20, Virtual Event
LFRR: Latent Factor Model RRS
Results:
• Tested against a large-scale real dataset with millions of users.
• Similarly promising performance to RCF (precision, recall and F1).
• Better efficiency: real-time recommendations under large datasets.
36. 26 July 2020, ACM SIGIR’20, Virtual Event
Hybrid RRS
Combine the strengths of more than one recommender technique
• CB combined with CF.
• CB combined with knowledge-based recommendation.
• CF combined with other approaches, e.g. Hidden Markov Models
(HMM) to predict next interaction based on k previous ones.
• Classical RS approaches combined with RRS (only when user-user
and user-item interactions co-exist).
Approaches
• Sequential/cascade, e.g. CF followed by HMM (Alanazi & Bain, 2016).
• Community detection to address cold-start problem. (Yu et al., 2018).
• Incorporate facial features (Zhang et al., 2017).
37. 26 July 2020, ACM SIGIR’20, Virtual Event
CCR: Content-Collaborative RRS
Key Elements
• Integrates distance metrics for CB and CF.
• CB: Distance between users based on profile attributes.
• CF: “Similar people [like / are liked by] and [dislike / are disliked by]
similar people”.
• Determine interaction groups for x based on the two principles.
J. Akehurst, et al. (2011). CCR: A Content-Collaborative Reciprocal Recommender for Online
Dating. Proceedings IJCAI’11.
CCR is the first RRS model that integrates CB and CF.
38. 26 July 2020, ACM SIGIR’20, Virtual Event
CCR: Content-Collaborative RRS
CCR Recommendation process
• Take interaction data associated to x, e.g. 𝐸𝑜𝐼VWXY 𝑥 , 𝐸𝑜𝐼ZX 𝑥
• Define the following interaction groups:
1. Users whom x likes.
2. Users who like x.
3. Users whom x dislikes.
4. Users who dislike x.
5. Users who reciprocally like x (intersection of 1 and 2).
• Apply CB first: determine set of users Sx with similar profile to x.
39. 26 July 2020, ACM SIGIR’20, Virtual Event
CCR: Content-Collaborative RRS
CCR Recommendation process
• Once we have Sx, apply CF by analysing user interactions:
• For every 𝑧 ∈ 𝑆%, determine list of users (s)he had reciprocal
interest with.
• This produces several lists of candidates v, one list for each z.
• Calculate each candidate’s support to finally rank them for x.
Sup(v,Sx) = #positive interactions - #negative interactions
Image source: Akehurst et al. (2011)
40. 26 July 2020, ACM SIGIR’20, Virtual Event
CCR: Content-Collaborative RRS
Results
• Evaluation against a baseline approach picking Sx at random.
• Success rate almost doubled (70%): % recommended users who
sent/received an EoI to/from x and response was positive.
• Cold-start problem alleviated.
• Memory-based à likely to be less efficient against large data.
Image source: Akehurst et al. (2011)
41. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
Key Elements
• User-to-user preference indicators and user-
to-content preference indicators coexist.
• Single-class RRS, no distinction between
different classes of users.
J. Neve, I. Palomares. (2020). Hybrid Reciprocal Recommender Systems: integrating item-to-user
principles in reciprocal recommendation. Companion Proc. ACM Webconf’20, pp. 848-854
HRRS combines principles from classical user-item recommendation
in the reciprocal recommendation process.
• Designed to connect users in platforms/apps where they publish
and share content à skill-sharing apps, e.g. Cookpad.
42. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
HRRS model
i. Item-to-user matching: Based on classical CF à users’ similarity on
content liked. User similarities on liked content is inherently symmetric.
ii. Reciprocal matching: Based on users’ preferences to other users
(requires reciprocity)
iii. Aggregation: Combination of both approaches into a final matching
between two users.
43. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
HRRS model: Item-to-user Matching
• Pairwise user similarities based on preferences towards content
published by any other users: bookmarked recipes.
• Unary ratings: preference by user x towards content c is either
“like” or “unknown” à Jaccard Index
• Problematic in domains with many contents items that are very
similar to each other.
• Example: user a liked “potato omelette” recipe, user b liked “Spanish
potato omelette”.
• Jaccard Index only accounts for co-occurrences of exactly the same
item.
44. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
HRRS model: Item-to-user Matching
• Solution: “Soft” extension of Jaccard index that captures
similar but not identical items that two users might have liked.
• Adjustment terms l,! introduce non-zero similarities
between non identical items.
• For quantifying text information (e.g. recipe titles and ingredients)
we use Word2Vec, producing word embeddings associated to
pairs of content items ra, rb
45. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
HRRS model: Reciprocal Matching
• Preference indicators: Follows to other users.
• Single-class Latent Factor Model (based on LFRR model).
• User x preference to user y properties.
• User y preference to user x properties.
• Harmonic mean to aggregate unidirectional user preferences into
reciprocal score.
Aggregation of user-user matchings
• The item-to-user and reciprocal matching scores m(x,y) are
aggregated using a simple weighted mean.
46. 26 July 2020, ACM SIGIR’20, Virtual Event
HRRS: A Hybrid classical-RRS
Results
• Offline evaluation with 500 user pairs from Cookpad website, 250 of
which indicated mutual preference through Follows.
• Predict a positive match (+) if the m(x,y) value returned by HRRS
is above a given threshold, and a negative match (-) otherwise.
• Two baselines: reciprocal only; content preference only.
48. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: measuring reciprocity
General RRS conceptual model
We are HERE!
49. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: measuring reciprocity
Reciprocity is arguably the most
fundamental and differentiating
complexity in RRS, with respect to
other RS.
The most commonly followed
approach to account for reciprocity
consists in calculating an
aggregated mutual preference
score…
… but it is not the only fusion
strategy to capture reciprocity. Let’s
have a look at some of them.
50. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: measuring reciprocity
Fusion methods for capturing reciprocity in RRS literature
Aggregation of numerical preference scores px,y and py,x
Harmonic mean, other means, sum, product, weighted mean, mixed aggregation…
Matrix multiplication
Set intersection of recommendation lists
Aggregation of probabilities
Conjuction (AND-like)
Community-level matching
Rank aggregation on recommendation lists
51. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Definition of Aggregation Function
𝒂(𝒙) = 𝒇(𝒂 𝟏(𝒙) , … , 𝒂 𝒏(𝒙)) , with f:[0,1]nà[0,1]
Basic Properties
f(0,…0) = 0 and f(1,…,1)=1 (Boundary)
x ≤ y implies f(x) ≤ f(y) with x = [x1 … xn] and y
= [y1 … yn] (Monotonicity)
In RRS, we are normally interested in the
particular case of two inputs:
f:[0,1]2à[0,1] = [0,1]x[0,1] à [0,1]
MAX
MIN
1
0
Disjunctive
(inc. t-conorms)
Conjunctive
(inc. t-norms)
Averaging
Optimistic
Pessimistic
NeutralA. Mean, Median
G. Mean,
H. Mean
52. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Weighted vs Non-weighted
• Importance weights assigned to
experts/criteria
Attitudinal Character of Aggregation
• Disjunctive vs Averaging vs Conjunctive (vs
Mixed)
• Optimistic vs Neutral vs Pessimistic
Choosing the right aggregation function is a
central aspect in any RS for combining
preferences, particularly in Group RS,
multicriteria RS and RRS.
MAX
MIN
1
0
Disjunctive
(inc. t-conorms)
Conjunctive
(inc. t-norms)
Averaging
Optimistic
Pessimistic
NeutralA. Mean, Median
G. Mean,
H. Mean
53. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Arithmetic mean (AM), Geometric mean (GM), Harmonic Mean (HM)
• AM strictly higher than GM, and GM strictly higher than HM, for non-
negative and non-identical inputs.
• In situations when inputs equal or close to 0, GM or HM may severely
affect the output (e.g. average mark of a student).
• GM popular in business and finance, e.g. to deal with percentages,
calculate growth rates, financial stock market indices, …
• HM is more stable to outliers, when there are very high (resp. low)
inputs. Useful when all inputs should be reasonably high to yield a
reasonably high output à matching people in RRS applications!
54. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Weighted means
The Weighted Arithmetic Mean allows for assigning importance
degrees to each of the elements to aggregate. Elements with highest
weights will be more influential in the aggregated output value.
The GM and HM also have their weighted counterparts:
If used in an RRS, how to
adequately weigh the two
parties’ unidirectional
preferences? Based on what?
55. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Conjunctive and disjunctive aggregation
T-norms are generalizations of the logical conjunction à conjunctive
aggregation functions (aggregation result below the minimum)
Examples:
-Minimum: T(a,b) = min(a,b)
-Product: T(a,b) = a*b
-Lukasiewicz t-norm: T(a,b) = max{0, a+b-1}
T-Conorms are generalizations of the logical disjunction à disjunctive
aggregation functions (aggregation result above the maximum)
Examples:
-Maximum: S(a,b) = max(a,b)
-Probabilistic sum: S(a,b) = a + b - a*b
-Bounded sum: S(a,b) = min{a+b, 1}
56. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Mixed-behaviour aggregation: uninorm functions
• A uninorm is a function U: [0, 1]2 → [0, 1] which is commutative,
associative, monotone, and has a neutral element gÎ [0,1]
• It fulfils the FULL REINFORCEMENT PROPERTY: two high (resp.
low) values reinforce each other, giving a higher (resp. lower)
aggregated output.
R.R. Yager. Uninorm Aggregation
Operators, Fuzzy Sets and Systems, 80(1),
pp. 111-120, 1996.
57. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Mixed-behaviour aggregation: uninorm functions
𝑝%,' 𝑝',% 𝑝%↔'
58. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Putting a few aggregation functions
together into an RRS
• Experiments with LFRR model.
• Dataset from Japanese dating site.
• Effect of using different aggregation
operators than the harmonic mean to
fuse pairs of users’ preferences.
Results
• Arithmetic mean outperforms in
recall.
• Cross-ratio uninorm doesn’t seem
very promising.
59. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Putting a few aggregation functions
together into an RRS
• Experiments with RCF model.
• Dataset from Japanese dating site.
• Effect of using different aggregation
operators than the harmonic mean to
fuse pairs of users’ preferences.
Results
• Cross-ratio outperforms HM in
precision and F1 score!
J. Neve, I. Palomares. (2019).
Aggregation Strategies in User-to-User
Reciprocal Recommender Systems. Proc.
IEEE SMC’19, pp. 4031-4036.
60. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: aggregation of
preferences
Sky is the limit…
61. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: matrix multiplication
• Used by (Jacobsen and Spanakis, 2019) in a system for matching
graduate students and jobs.
• Define a student-course matrix A, where element grades,c is the
grade obtained by student s in course c.
• Define a job-course matrix B, where element simj,c describes the
similarity between the characteristics of job j and course c contents.
• Each element of the product ABT is the matching between student s
(row vector in A) and job j (column vector in B)
62. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: weighted mean
• Used by (Kleinerman et al., 2018), where the mutual preference
score is calculated as:
𝑝%↔' = 𝛼% ⋅ 𝑝%,' + 1 − 𝛼% 𝑝',%
• The weighting parameter 𝛼% ∈ [0,1] is optimized for each subject
user x, namely for optimizing successful interactions.
• In other words, finding an optimal balance between subject user
and object user that maximizes successful interactions in the
recommendations received.
63. 26 July 2020, ACM SIGIR’20, Virtual Event
Fusion process: recommendation
rankings
• Used by (Mine et al., 2013), in a job matching RRS.
• Let si be a job seeker and rj a recruiter.
• Two unidirectional recommendation lists are created, one for the
job seeker and one for the recruiter.
• The positions assigned to si in the recommendation list for rj,
and vice versa, are aggregated to yield the matching score:
𝑟𝑎𝑛𝑘(𝑠r(𝑟s)) 𝑟𝑎𝑛𝑘(𝑟s 𝑠r ) 𝑚𝑎𝑡𝑐ℎ𝑖𝑛𝑔
1 1 1
1 2 0.5
1 5 0.2
2 3 0.167
3 4 0.083
65. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(1) Online Dating
• Most popular target domain for RRS
innovations.
• Earliest on RS for online dating, from
around 2007 à Nonreciprocal
• First reciprocal approaches, primarily
CB (2010-2013).
• Gentle shift towards CF and hybrid
afterwards.
Image source: @graylab
66. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(1) Online Dating – theoretical studies
• Comprehensive case studies upon
several success and evaluation metrics.
• Sensitivity to detect scammers.
• Temporal behavior, messaging and
replying patterns of users.
• User correlations under different
attributes.
• Deviation between explicit preferences
and implicit preferences inferred from
behavior.
Image source: @graylab
67. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(1) Online Dating – theoretical studies
• Finding “invisible communities” from
messaging graphs, with clusters defined
by homophily and attractiveness.
• Potential of facial attractiveness
information mined from user pictures to
overcome sparsity.
• Gender attribute differences in decision-
making to choose a date.
• Computational complexity analyses.
Image source: @graylab
68. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(1) Online Dating – models/techniques
• Preferences as frequency distributions
of attributes’ values (RECON).
• Hidden Markov Models to predict next
interaction(s) based on past ones.
• Explicit preferences from
questionnaires.
• Deep learning on social media text.
• Bipartite network (RCF).
• Clustering similar users.
• Tensor models of user attributes-
interactions.
• … Image source: @graylab
69. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(2) Recruitment
• Scenarios where a job seeker looks
for recommendations, and both
her/his interests and the recruiter
interests need to align.
• Less RRS than in online dating, but
good diversity of strategies.
• A few works focused on graduate
students’ recruitment.
Image source: @huntersrace
70. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(2) Recruitment – theoretical studies
• Characterization of online
recruitment services and
monitoring demand-offer of
employment.
• Correlations between explicit and
implicit job preferences.
• Identifying best indicators of job
seekers’ preferences (implicit).
• Impact of reciprocity versus
nonreciprocity.
Image source: @huntersrace
71. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(2) Recruitment – models/techniques
• Analyzing past successful
graduates for reciprocal graduate-
job recommendation.
• Binary classification to predict
clicks on jobs.
• Privacy oriented stable matching.
• Walrasian Equilibrium multi-
objective optimization à fairness.
• Deep matrix factorization. Image source: @huntersrace
72. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(3) Online Learning
• Often large and diverse body of
learners.
• Increasingly popular (or needed…).
• Predominantly CB approaches based on
exploiting learners and teachers’ profile
information.
• Variety of scenarios:
• Peer matching in MOOCs and
university courses.
• Group formation.
• Learner-question matching in forums.
• Student-supervisor matching.
Image source: @andyfalconerphotography
73. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(3) Online Learning – theoretical studies
• Effects of peer recommendation in learner
engagement.
• Study of recommendation strategies in
MOOCs.
Models
• Peer matching based on RECON (attribute
frequency distributions).
• Peer matching based on compatibility
criteria.
• Group formation via optimization from
individual recommendation lists.
• CB-CF for student-supervisor allocation. Image source: @andyfalconerphotography
74. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(4) Online Social Networks
• Social network sites can be symmetric
or asymmetric.
• Symmetric SNSs require both sides to
confirm the relationship (e.g.
Facebook, LinkedIn) à RRS
• Asymmetric SNSs: one user can follow
or connect to another user, without a
requirement the other way round.
• Asymmetric SNS tend to show much
more skewed in-degree and out-
degree distributions.
Image source: @jtylernix
75. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(4) SNSs – theoretical studies and
models
• Explanations in the success of people
recommendation.
• Recommendation of unfamiliar people
(strangers).
• Network structure characteristics
(distribution, centrality, etc.).
• Trust and reputation-based
recommendation.
• Proximity-driven link prediction.
• Genetic algorithms on graphs.
Image source: @jtylernix
76. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Applications
(5) Other Emerging Domains
• Socializing: Finding people to
meet outside the Internet e.g. to
practice a hobby.
• Skill-sharing: Sharing skills and
learning from other users’ shared
content.
• Shared economy: Peer-to-peer
activity, based on collaboration.
• Mentor-mentee matching
• Scientific collaboration
Image source: @dylandgillis
Image source: @cytonn_photography
78. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Measuring success
• High cost of online evaluation with live users.
• Important to perform a good offline evaluation with historical data.
“For reciprocal recommenders, the evaluation can also use different levels of success. For
instance, a job recommendation can be seen as somewhat successful when the user (as a subject)
decides to apply for the recommended job; the same recommendation is more successful when
the same user is called to be interviewed for the recommended job (the object); and even more
successful when the user is selected for the position.”
L. Pizzato, et al. (2013). Recommending people to people: the nature of reciprocal recommenders with
a case study in online dating. User Modelling and User-Adapted Interaction, 23, pp.447-488.
79. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Measuring success
• Defining a general, realistic success metric requires quantifying the
importance of all the factors that may influence such success.
• In a job recommender, this measure needs to:
• Assess the importance of a job seeker being selected for an
interview, with respect to…
• Assessing the importance of applying for a recommended job.
• Both aspects are important, but maybe not equally.
• The above is difficult to define, hence usually RRS are evaluated
using more specific, fine-grained success metrics.
80. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Measuring success – evaluation metrics
• Like: whether a subject user likes or not a recommended user.
• Useful to explain user behavior patterns.
• Like-back: whether a subject user likes an object user whom also
likes the subject user à Reciprocated recommendation.
Using either of the above two as successful when it occurs, we can
define a number of metrics, given a list of N recommendations:
Precision at N: 𝑷@𝑵 =
#𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
𝑵
𝑷@𝑵 = 𝟑/𝟖
81. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Success rate at N: S@𝑵 =
#𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
#𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍E#𝒖𝒏𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
=
𝟑
𝟑E𝟐
=
𝟑
𝟓
Unsuccessful recommendations could include:
• EoI sent by the subject user and not replied by the object user.
• EoI sent by the subject user and replied negatively.
Failure rate at N: S@𝑵 =
#𝒖𝒏𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
#𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍E#𝒖𝒏𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
=
𝟐
𝟑E𝟐
=
𝟐
𝟓
In some domains like online dating, minimizing failure (false positive rate)
might be even more important than maximizing success and precision
(true positive rate).
82. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Recall at N: How close a recommendation list is to contain all the
known successful interactions (ground-truth) involving x.
R@𝑵 =
#𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍
#𝒌𝒏𝒐𝒘𝒏 𝒔𝒖𝒄𝒄𝒆𝒔𝒔𝒇𝒖𝒍 𝒊𝒏𝒕𝒆𝒓𝒂𝒄𝒕𝒊𝒐𝒏𝒔
• If test data (interactions to be predicted for x) include a total of 6
successful interactions, and three of them have been “predicted” as part
of the recommendation list of size N=8, then R@8 = 3/6=0.5
F1 measure: Combines precision and recall. 𝐹1 =
_ ˆ⋅‰
ˆE‰
83. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
Discounted Cumulative Gain at N:
A measure of ranking quality à It takes the ranking position of
recommended users y in the recommendation list into account.
𝐷𝐶𝐺@𝑁 = ∑rŽ^
• _•‘’“”^
•–—˜(rE^)
=
^
•–—˜(™E^)
+
^
•–—˜(šE^)
+
^
•–—˜(›E^)
= 4.12
• reli =1 if the ith recommendation in the list is relevant
(successful), and 0 otherwise.
84. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
GENERAL EVALUATION PIPELINE
1. Split user-user interaction data in the form <user1, user2, interaction>
(where interaction is a single atomic interaction from user1 to user2 ) into
training and test data.
85. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
GENERAL EVALUATION PIPELINE
1. Split user-user interaction data in the form <user1, user2, interaction>
(where interaction is a single atomic interaction from user1 to user2 ) into
training and test data.
2. Define what success will represent: liked recommendation vs reciprocated
recommendation.
86. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
GENERAL EVALUATION PIPELINE
1. Split user-user interaction data in the form <user1, user2, interaction>
(where interaction is a single atomic interaction from user1 to user2 ) into
training and test data.
2. Define what success will represent: liked recommendation vs reciprocated
recommendation.
3. Select evaluation metric(s) to use – what aspects are relevant in our
domain?
87. 26 July 2020, ACM SIGIR’20, Virtual Event
RRS Evaluation
GENERAL EVALUATION PIPELINE
1. Split user-user interaction data in the form <user1, user2, interaction>
(where interaction is a single atomic interaction from user1 to user2 ) into
training and test data.
2. Define what success will represent: liked recommendation vs reciprocated
recommendation.
3. Select evaluation metric(s) to use – what aspects are relevant in our
domain?
4. Take a set of tests users x, specify a recommendation list N, execute our
RRS and average for all users the results obtained in the metric(s) used.
89. 26 July 2020, ACM SIGIR’20, Virtual Event
Common RRS Challenges
Popularity Bias
• Popular users in an RRS are those liked by an unduly large of
other users in the system.
• If not dealt with properly, the presence of very popular users
can negatively affect not only themselves, but also less
popular users:
• Popular users may receive an overwhelming number of
recommendations, most of which may not be of relevance to them,
which may decrease their satisfaction.
• By contrast, unpopular users may be neglected by the RRS, or even
suffer a negative experience by being often recommended to
popular users who won’t reciprocate.
90. 26 July 2020, ACM SIGIR’20, Virtual Event
Common RRS Challenges
Popularity Bias
Solutions to manage popular users and prevent ‘bias’:
• Identify communities of recommendable users around
popular users too “split the load”.
• Ensure that every user receives as many recommendations as
number of times they will be recommended to other people.
• Weighting (balancing) the relative importance of x and y in
the calculation of reciprocal preference (as seen earlier).
91. 26 July 2020, ACM SIGIR’20, Virtual Event
Common RRS Challenges
Fairness and Explainability
• Ensuring as equal opportunities as possible to all users or
communities of them.
• Preventing discrimination, mistreatment and inequality
issues, specially with vulnerable groups. (B. Xia et al. 2019, WE-Rec)
• Explainability has been barely investigated in RRS. The
following work provides a “starting point”:
“Are we able to derive any insight on how these models are learning to recommend
relationships? Are attention models able to produce explainable relationship
recommendations?” (Tay et al. 2018,CoupleNet)
92. 26 July 2020, ACM SIGIR’20, Virtual Event
The long way ahead
93. 26 July 2020, ACM SIGIR’20, Virtual Event
Conclusion
KEY MESSAGES
1. Reciprocal Recommenders recommend people to connect with, introducing the
requirement of satisfying the recommended user, not only the end user receiving
the recommendation.
2. Online dating is the most prominent application domain, followed by recruitment,
online learning and symmetric social networks.
3. Some domains have predominantly content-based approaches, with collaborative
filtering and hybrid ones being investigated more recently.
4. The fusion of information (preferences, recommendation lists, etc.) is crucial to
measure and integrate reciprocity in recommendation, and increasing success.
5. There are as many pending challenges as opportunities to expand the scope of RRS
research.
94. 26 July 2020, ACM SIGIR’20, Virtual Event
Thank you
Reciprocal Recommendation
Matching Users with the Right Users
Contact: ivanpc(at)ugr(dot)es