CIKM 2022 Paper: https://arxiv.org/abs/2208.09864 GitHub: https://github.com/joisino/consul

1. 1 KYOTO UNIVERSITY
KYOTO UNIVERSITY
Towards Principled User-side Recommender Systems
Ryoma Sato

2. 2 KYOTO UNIVERSITY
What can we do when the system is unsatisfactory
Solution？
The Recommender system on Twitter is unsatisfactory
Then, stop using Twitter
I want to use Twitter
Then, request a new recommender system to Twitter
It’s hopeless, and it takes long time even if it succeeds

3. 3 KYOTO UNIVERSITY
What can we do when the system is unsatisfactory
Solution
The Recommender system on Twitter is unsatisfactory
Then, stop using Twitter
I want to use Twitter
Then, request a new recommender system to Twitter
It’s hopeless, and it takes long time even if it succeeds
Solution: user-side recommender systems

4. 4 KYOTO UNIVERSITY
User-side RecSys is realized by users

User-side RecSys is a RecSys that a user of
the service builds/realizes

It is in contrast to standard RecSys,
which are realized by the employed engineers

It’s completely different from steerable RecSys

Steerable RecSys are implemented by the employed
engineers, so users need to wait until implemented
 Steerable Recsys enables
users to select interests
Active Solution
Passive Solution

5. 5 KYOTO UNIVERSITY
Challenge: users cannot access to the database

Standard RecSyS
are implemented by Engineers of
Twitter, who have complete
access to the database.

User-side RecSys
are implemented by users, who don’t have direct access to the database.
Twitter Inc.
Database
Twitter Engineer
RecSys
SQL
ANN
Twitter Inc.
Database
User
SQL
Search Query
API

6. 6 KYOTO UNIVERSITY
We consider item-to-item recommendation

Problem setting:
item-to-item recommendation
Movie RecSys @IMDb
Item: movie
Users who watched Toy
Story also watched them.
User RecSys @Twitter
Item: user
Users who follow
@SIAMDataMining also
follow them.
Product RecSys @Amazon
Item: product
Customers who bought this
item also bought
https://www.imdb.com/title/tt0114709/ https://twitter.com/SIAMDataMining https://www.amazon.com/dp/0262035618

7. 7 KYOTO UNIVERSITY
Are web pages enough for building RecSys?

Users observe little information for bulding RecSys
Users observe web pages only
(and estimate information indirectly)
Official developers observe and manipulate DB
Thus they can easily build a RecSys
RQ1: Are these information enough for
users to build RecSys?

8. 8 KYOTO UNIVERSITY
Users can reverse engineer item representations

RQ1: Are these information enough for users to build RecSys?

A: Yes!!
I showed that users can ``reverse-engineer’’ the
representations of items, which are sufficient for building
Recsys.

Idea: The recommendation graph is observable.
This is the k-NN graph of item representations.
The representation can be estimated from kNN graphs
via manifold learning
Official recommendations
build
Recommendation graph

9. 9 KYOTO UNIVERSITY
Use-side Recsys is feasible, but expensive

The result indicates that building a user-side recommender
system is a feasible problem from information-theoretic limit.

Naive user-side recommender algorithm:
1. Visit all item pages
2. Build the recommendation graph
3. Estimate the item representations
4. Build and customize a recommender system based
on the representations
(e.g., by filtering categories and lengthening the slot)

Problem: Too much cost

RQ2: Can we sidestep the direct reverse engineering
and build RecSys more efficiently?

10. 10 KYOTO UNIVERSITY
We want each sensitive attribute to appear equally

Assume that each item has a discrete sensitive attribute and that
the sensitive attributes of the items are observable.

We want each sensitive attribute to appear equally in the results.

What kind of sensitive attribute to use is at the discretion of us.
Movie:
Sensitive attribute = country
User: Sensitive attribute = gender
https://twitter.com/Suzu_Mg https://twitter.com/sudaofficial
https://www.imdb.com/title/tt0114709/
https://www.imdb.com/title/tt0245429/
Another example:
Sensitive Attribute = Popularity ∈ {High, Mid, Low}
（Based on the number of followers)
woman
woman
woman woman
man
man

11. 11 KYOTO UNIVERSITY
We want a user-side RecSys with 3 good props

I propose three desirable properties:

Consistency: If we do not impose fairness, nDCG of a
user-side RecSys should be the same as the official one.

Soundness: If we specify τ, a user-side RecSys should
show at least τ items from each sensitive attribute.

Locality: A user-side RecSys should not download all
pages.

The existing methods and the reverse engineering
approach lack at least one property.

RQ2’: Is there a user-side RecSys that satisfies all?

12. 12 KYOTO UNIVERSITY
I propose a user-side RecSys with three props

RQ2’: Is there a user-side RecSys that satisfies all?

A: Yes!!
I propose Consul, a user-side RecSys that satisfies
all three properties.

Idea: Consul also uses the recommendation graph.
But it traverses the graph locally via DFS.
The official system is
biased to American movies.
I want to
recommend movies
for Toy Story with countries in mind.

13. 13 KYOTO UNIVERSITY
Experiments

14. 14 KYOTO UNIVERSITY
Consul’s recommendations are diverse

Case Study:

I set release dates as the sensitive attribute and built a
user-side movie recommender system with Consul.

The official one recommends contemporary popular
movies.
Consul recommends old sci-fi movies as well (diverse).
The recommendations for
Terminator 2.

15. 15 KYOTO UNIVERSITY
Consul performs well

Quantitative Experiments:

I evaluated existing methods and Consul with standard
benchmarks

The proposed method performs as well as existing
methods
(proposed) →

16. 16 KYOTO UNIVERSITY
Consul is much efficient

Quantitative Experiments:

I evaluated existing methods and Consul with standard
benchmarks

The proposed method is much more efficient than
existing methods thanks to locality
(proposed) →

17. 17 KYOTO UNIVERSITY
Consul is much efficient

User study:

I asked which of the official recommender systems and
the user-side system build by Consul provide more
relevant and diverse recommendations.

About the same number of users chose the official one
and Consul in terms of relevance.
Many users thought Consul was more diverse.
Which system is more relevant?
Which system is more diverse?

18. 18 KYOTO UNIVERSITY
Consul is much efficient

Case study in the wild:

I conducted a case study on real-world Twitter.

I built use-side user recommendation systems.

I’m not an employer of Twitter but an ordinary user.
Nevertheless, I succeeded in building a new RecSys on
Twitter.
Results for Hugh Jackman
I set gender as the sensitive attribute
Consul recommends three men and women
Consul is much efficient (real-time inference)

19. 19 KYOTO UNIVERSITY
Conclusion

20. 20 KYOTO UNIVERSITY
Conclusion

I answered two RQ for user-side RecSys.

RQ1: Are available information enough for users to build
RecSys?
→ Yes, by ``reverse engineering’’ representations.

RQ2: Is there a user-side RecSys that satisfies all the
three properties?
→ Yes. I propose Consul.

The proposed method performs well and is much more
efficient than existing methods.
arXiv: https://arxiv.org/abs/2208.09864 GitHub: https://github.com/joisino/consul