IntRS 2025 presentation at RecSys 2025. Abstract: Recommender Systems (RecSys) have transformed personalized applications by delivering tailored content and experiences. However, modern Deep Learning RecSys often operate as opaque “black boxes,” offering users no control over how personalization is shaped. We introduce a novel algorithmic approach to bridge this gap in the context of visual art recommendation by integrating user agency directly into the RecSys engines. By allowing users to dynamically adjust facets such as content diversity and popularity, through the use of hyperparameters implemented as sliders, the system creates a feedback loop where users can actively tune recommendations while also helping the system to learn about their preferences. This approach ensures that personalization is not only algorithmically optimized but also user-driven, fostering a balance between automation and human control. The results of a large-scale user study (n=151) evidenced that sliders enhance engagement and recommendation quality by promoting meaningful exploration.

1. ArtEx: An Interactive Visual Art Recommendation
with Diversity and Popularity Control
Bereket A. Yilma, Rully Agus Hendrawan, Peter Brusilovsky, and Luis A. Leiva

2. Human-AI collaboration and User Control in Recommender Systems
Two brains are better then one!

3. • User Profile (TasteWeights, Grapevine)
• Choice of peers (PeerChooser)
• Fusion of sources (SetFusion, RelevanceTuner)
• Important parameters - popularity, diversity
Past Exploration of User Control and Human-AI Collaboration

4. TasteWeights: User Profile Control
Knijnenburg, Bart P., Svetlin Bostandjiev, John O'Donovan, and Alfred Kobsa. 2012. "Inspectability and Control in Social
Recommenders." In 6th ACM Conference on Recommender System, 43-50. Dublin, Ireland.

5. PeerChooser: Controlling Peers
O'Donovan, John, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. "PeerChooser: visual interactive
recommendation." In Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1085-88. Florence, Italy.

6. PeerChooser: Controlling Peers
Tsai, C.-H. and Brusilovsky, P. (2021) The effects of controllability and explainability in a social recommender system. User Modeling and
User-Adapted Interaction 31 (4), 591–627.

7. • Some parameters of the recommendation
process are very hard for a RecSys to guess
• These parameters can also be changing
between and within sessions
• But in a number of cases, it is easy for the
users to contribute
• Do not guess, ask the user!
What is the Message?

8. Personalised
Recommendation
Task
8
Model
Training
R
𝑚 × 𝑚
● If a user likes painting A find paintings B, C, D that are similar to A.
Good representation of
the data!
Visual Art Recommendation

9. Diego
Velasquez
Nude art of
17th century
Baroque
painting
122,5 x 177
cm
Oil canvas
Marquis del
Carpio
Room 30 of
National Gallery
of London
Art History
Style
Material
History
Authorship
Location
Size
Early approaches in VA RecSys:
➔ Manually curated metadata to drive recommendations.
Visual Art Recommendation

10. Modern approaches in VA RecSys:
➔ Train a model that can learn Visual features from images of
paintings to drive recommendations
➔ Use pre-trained models as feature extractors
- ResNet, AlexNet, GoogLeNet, VGG
● Recommendations don’t have direct
interpretation.
Visual Art Recommendation
Latent semantic representations

11. Modern approaches in VA RecSys:
➔ Train a model that can learn latent semantic relationships of Paintings and textual descriptions
of paintings to drive recommendations
Visual Art Recommendation
Unimodal Multimodal
Contrastive
Language-Image
Pre-training (CLIP)
Bootstrapping
Language-Image Pre-
training (BLIP)
Yilma et al. UMAP 2023
Yilma et al. CHI 2023

12. Visual Art Recommendation
➔ Large amounts of content: location / topic -specific
➔ Low ratio of users to item
➔ Cold start challenges for new items
Content-based techniques
➔ Discourage exploration and overshadow
lesser-known items (Diversity)
Popularity and diversity are both critical factors

13. User Profile
Preferences
Pu
= {P1 , P2 , . . . , Pn} ; Pu ∈ P
Ratings Wu = {w1 , w2 , . . . , wn}
Unimodal Engines
Image based Text based
ResNet VGG LDA BERT
Multimodal Engines
CLIP
BLIP
FLAMINGO
R
𝑚 × 𝑚
Learned embeddings Similarity Matrix
S
u (Pi )
Model Score
Black Box RecSys Engines
.
.
.
.
.
.
.
.
.
.
Ranked list of
Recommendations
based on Model
Score
PR= {P1 , P2 , . . . , Pr}
.
.
.
.
.
Popularity and diversity are both critical factors

14. Will it make sense to give control over popularity and diversity in art recommendation systems to the user?
● A recommender system could use large user data to choose “optimal overall” parameters
● Curators might want to balance iconic and lesser-known artworks and expose users to a diverse set of
paintings (by age, genre, material, etc)
● Users may prefer mainstream vs niche and divers vs focused content depending on context
Who Should Set Up Popularity and Diversity Parameters?
Curator-visitor tradeoff

15. ArtEx System Overview

16. ArtEx: Recommendation Scoring
1. Adjusting Popularity:
● We introduces a popularity score S(p,Pop) for all the paintings in the dataset. derived from artwork rankings of
popular artists within the SemART dataset, based on public reviews reflecting the collective opinion on notable
artworks.
S(P ) = 𝞪 S(P ,U) + 𝞫 S(P , Pop)
● 𝞫 is user provided hyper parameter determining user's interest to see popular items. 𝞪 = 1- 𝞫

17. ➢ We define a fair painting selection function Ψ(R).
➢ The function Ψ(R) rewards a typical diversity of stories in the recommendation set.
2. Adjusting Diversity:
● Si , i = 1,...K is the story-partition of the dataset.
● R is the recommendation set
● 𝛄p is a representativeness score of painting group carried by painting p in the
recommendation set.
Group 1 Group 2 Group 3
+
ArtEx: Recommendation Scoring
Representative & Informative
Query Selection Yilmaz et al.
SIGIR2015.

18. 3. Adjusting Multi-Dimensional Diversity:
ArtEx: Recommendation Scoring

19. ArtEx: Recommendation Strategy

20. ArtEx Study: Three User-Control Options
→ Test how granularity of control affects user engagement
Ratings-based
recommendations

21. ArtEx: User Study
for condition 2 and 3
(with sliders)

22. System: UI
23

23. ● Actions tracked:
○ Rate, Collect, Remove, Lookup, Use Sliders
● No significant differences across conditions
Results: User Interaction

24. High-Slider-Use Cohort Works More Efficiently
30

25. High-Slider-Use Group Works More Efficiently
31

26. High-Slider-Use Group Works More Efficiently
32

27. Why High-Slider-Use Group Works More Efficiently?
33
Low-slider-use cohort wastes a lot of efforts “pushing down” a clutter of bad recommendations

28. Results – Slider Efficiency
● High slider users rated fewer items but still found
enough 5★ paintings
● Low slider users relied on many 'mediocre' ratings
(2–3★ items) to improve recommendations
● Positive correlation between slider use and
efficiency (ratio of 5★ items) in reaching good
recommendations
35

29. Design Implication: Slider Use Matters
● Rating item with 2-4★ is a “poor-man”control
mechanism to push down bad items and bubble
up better ones
● Direct control using sliders brought good items to
the surface faster → more efficient search
● High slider users = more efficient work, fewer
ratings, higher ratio of 5★ ratings
36

30. ● Users control might transforms recommendation from a black-
box choice of recommendation parameters into a human-AI
collaborative exploration
○ Sliders reduce effort: fewer ratings needed to achieve same
outcome
○ Users with granular controls (6-Sliders) showed highest
engagement and discovery potential
● User control might improve the efficiency of user-RecSys work
Conclusion

31. Try the ArtEx Web App
or Join the Study
Read the paper
ArtEx: An Interactive Visual Art Recommendation
with Diversity and Popularity Control
Bereket A. Yilma, Rully Agus Hendrawan, Peter Brusilovsky, and Luis A.
Leiva
Read Our Paper! See our RecSys 2025 Demo on Thursday! Try ArtEx!