This document analyzes popularity bias in book recommendation algorithms. It finds that most recommendation algorithms disproportionately recommend popular books, harming recommendations for niche users who prefer less popular books. It evaluates algorithms based on how they balance personalization against fairness of exposure across popular and niche items. While algorithms differ in their ability to capture individual tastes, addressing popularity bias could improve recommendations for all user groups, especially niche users who receive lower quality recommendations under current algorithms.

1. The Unfairness of Popularity Bias in Book Recommendation
Hossein A. Rahmani
WI Group
University College London
h.rahmani@ucl.ac.uk
Mohammadmehdi Naghiaei
DECIDE
University of Southern California
naghiaei@usc.edu
Mehdi Dehghan
Abin's Lab
Shahid Beheshti University
mahdi.dehghan551@gmail.com
Third International Workshop on Algorithmic Bias in Search and Recommendation (Bias 2022)
The 44th European Conference on Information Retrieval (ECIR 2022)
April 10, 2022, Stavanger, Norway

2. Popularity Bias in RecSys
● Popularity Bias
● Evaluation Metrics in Popularity Bias:
○ Long-tail exposure
○ Calibration
Reading distribution of books.
2
The concept of Calibration (Credit by [1])
[1] Abdollahpouri, Himan, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. "The unfairness of popularity bias in
recommendation." arXiv preprint arXiv:1907.13286 (2019).

3. Domain
● Book-Crossing Dataset.
● Why Book domain?
● Statistics of Book-Crossing dataset.
#Users 6358
#Books 6921
#Interactions 88552
13.92
12.79
Sparsity 99.80%
Time Span August-September 2004
3

4. RQ1
How much are different individuals or groups of users
interested in popular books?

5. Popularity Bias in Book-Crossing Dataset: Dataset Observations
Correlation of user profile size and the popularity of books in the user profile.
5

6. Popularity Bias in BookCrossing Dataset
Dataset Observations
● Categorizing Users:
○ Niche users
○ Diverse users
○ BestSeller-Focused users
● Average profile size for
different user groups:
6

7. RQ2
How does the popularity bias in recommendation
algorithms impact users with different tendencies toward
popular books?

8. Reproducibility Setup
Evaluation Metrics
● Precision
● Recall
● MAE
● nDCG
● ΔGAP
Base Recommendation Models
● Baseline approaches:
○ Random
○ MostPop
○ BPR
● KNN approach:
○ User KNN
● Neural Network approaches:
○ NeuMF
○ VAECF
● Matrix Factorization approaches:
○ MF
○ PMF
○ NMF
○ WMF
○ PF
8

9. Recommendation of Popular Books
BPR NeuMF
NMF
PF
MF
VAECF UserKNN
WMF
The correlation between the popularity score of items and the number of times they are being recommended by
using base recommendation algorithms.
9
Random MostPop
PMF

10. Popularity Bias on Different User Groups: ΔGap
The Group Average Popularity (∆GAP) of different algorithms for Niche, Diverse, and Bestseller-focused user groups.
10

11. The performance of base models for the three user groups in terms of MAE (the lower, the better) and Precision, Recall,
and NDCG (the higher, the better).
Popularity Bias on Different User Groups: Other Metrics
11

12. The correlation between NDCG and ∆GAP for the three user groups.
Unfairness of Popularity Bias vs. Personalization
12

13. Conclusion and Future work
● Different users have a considerably different tendency towards popular items
● Most state-of-the-art recommendation algorithms are providing significantly lower recommendation
quality to Niche and Diverse users despite having a larger profile size
● Propagation of popularity bias affects all user groups but to a significantly different magnitude
● Algorithms could differ significantly in their ability to capture users’ tastes based on the domain
● An underlying trade-off seems to exist between personalization and fairness of popularity bias

14. Thank you!
Mohammadmehdi Naghiaei, Hossein A. Rahmani, Mahdi Dehghan
@naghiaei, @srahmanidashti
https://github.com/rahmanidashti/FairBook
https://rahmanidashti.github.io/FairBook/