This document discusses adversarial attacks on recommendation systems, detailing various attack methodologies including data poisoning and user profile manipulation. It categorizes attacks into white-box, grey-box, and black-box types, highlighting the challenges and strategies for each approach. The content emphasizes the importance of understanding these attacks to defend against potential risks in recommender systems.

1. Adversarial Attacks for
Recommendations
1
Data Science and Engineering Lab
Tutorial website: https://advanced-recommender-systems.github.io/ijcai2021-tutorial/
Wenqi Fan
The Hong Kong Polytechnic University
https://wenqifan03.github.io, wenqifan@polyu.edu.hk

2. Adversarial Attacks on Deep Learning
Classified as panda
𝑥
Small adversarial noise
𝜖
Classified as gibbon
𝑥′

3. 3
Attacks can happen in Recommender Systems
https://www.bbc.com/news/business-47941181
https://www.gov.uk/government/news/facebook-and-ebay-pledge-to-combat-trading-in-fake-reviews
“More than three-quarters of
people are influenced by
reviews when they shop online.”
Defend against potential
adversarial attacks
Understand how attacks
can be performed

4. 4
Attacks can happen in Recommender Systems
• Security (Attacking) in Recommender Systems
• Data poisoning/shilling attacks: promote/demote a set of items
Adversary
(fake user profiles)

5. ...
...
... ... ... ... ...
...
...
...
Normal
Data
Fake
User Profiles
Target/Victim
Recommender System
User-item Interactions
Target items being recommended
Generating
Fake
User
Profiles
Attacker
A General Attacking Framework
5

6. Attack settings
6
qWhite/grey-box attacks vs. Black-box attacks.
• have full/partial knowledge of the victim model/have no
knowledge.
qTargeted Attacks vs. Non-Targeted Attacks.
• attack specific target items / hurt the overall recommendation
performance.
Grey-
box
White-
box
Black-
box
Adversary’s Knowledge
High Low

7. § White-box Attacks
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering (NIPS’16)
§ Grey-box Attacks
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
(RecSys’20)
• Adversarial Attacks on an Oblivious Recommender (RecSys’19)
§ Black-box Attacks
• CopyAttack: Attacking Black-box Recommendations via Copying Cross-domain
User Profiles (ICDE’21)
• PoisonRec: An Adaptive Data Poisoning Framework for Attacking Black-box
Recommender Systems (ICDE’20)
Adversarial Attacks
7

8. § White-box Attacks
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering (NIPS’16)
§ Grey-box Attacks
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
(RecSys’20)
• Adversarial Attacks on an Oblivious Recommender (RecSys’19)
§ Black-box Attacks
• CopyAttack: Attacking Black-box Recommendations via Copying Cross-domain
User Profiles (ICDE’21)
• PoisonRec: An Adaptive Data Poisoning Framework for Attacking Black-box
Recommender Systems (ICDE’20)
Adversarial Attacks
8

9. § Collaborative Filtering:
§ Given data. ,
§ Goal: matrix completion
§ Alternating minimization:
Preliminaries
9
Data Poisoning Attacks on Factorization-Based Collaborative Filtering, NIPS, 2016

10. § Inject malicious users
§ The CF formulations will be:
§ Goal：
§ Solution: Projected gradient ascent (PGA)
Attacking Formulation
10
Data Poisoning Attacks on Factorization-Based Collaborative Filtering, NIPS, 2016

11. § White-box Attacks
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering (NIPS’16)
§ Grey-box Attacks
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
(RecSys’20)
• Adversarial Attacks on an Oblivious Recommender (RecSys’19)
§ Black-box Attacks
• CopyAttack: Attacking Black-box Recommendations via Copying Cross-domain
User Profiles (ICDE’21)
• PoisonRec: An Adaptive Data Poisoning Framework for Attacking Black-box
Recommender Systems (ICDE’20)
Adversarial Attacks
11

12. Threat Model
Attacker’s Goal: promote certain items availability of being recommended
Attacker’s knowledge: fully (partial) observable dataset
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.

13. § Step 1: Train surrogate model
How to attack a RecSys: A bi-level optimization problem
Training Recommender System
Normal Data Injected fake data
where $
𝑋 is the fake rating matrix, 𝜃∗
is the parameters of the
surrogate model
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.

14. § Step 2: Evaluate the malicious goal after fake data are consumed
How to attack a RecSys: A bi-level optimization problem
Adversarial objective
(defined on prediction on normal data)
Well-trained surrogate model parameters
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.

15. Solving the bi-level optimization: gradient-based
Train surrogate model
based on new fake data
Obtain gradient and
update fake data
Repeat until
converge
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.

16. Limitations
How to obtain the desired gradients
Lacking exactness in gradient computation
ignored
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.

17. § Exact Solution
Computational graph
Inner objective Outer objective
Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys 2020.
Forward computation flow: solid black arrow
Gradient backpropagation flow: dashed red arrow

18. § White-box Attacks
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering (NIPS’16)
§ Grey-box Attacks
• Revisiting Adversarially Learned Injection Attacks Against Recommender Systems
(RecSys’20)
• Adversarial Attacks on an Oblivious Recommender (RecSys’19)
§ Black-box Attacks
• CopyAttack: Attacking Black-box Recommendations via Copying Cross-domain
User Profiles (ICDE’21)
• PoisonRec: An Adaptive Data Poisoning Framework for Attacking Black-box
Recommender Systems (ICDE’20)
Adversarial Attacks
18

19. § Challenges in existing attacking methods:
§ Less "realistic" user profiles (easily detected)
Challenges
19
...
...
... ... ... ... ...
...
...
...
Normal
Data
Fake
User Profiles
Target/Victim
Recommender System
User-item Interactions
Target items being recommended
Generating
Fake
User
Profiles
Attacker
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

20. § Cross-domain Information
§ Share a lot of items
§ Users from these platforms with similar functionalities also share similar
behavior patterns/preferences.
Solution
20
Taobao JD.com
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

21. § Challenges in existing attacking methods:
§ Less "realistic" user profiles (easily detected)
• Copy cross-domain users with real profiles from other domains
Solution
21
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

22. § Challenges in existing attacking methods:
§ Less "realistic" user profiles (easily detected)
• Cross-domain Information
§ White/Grey-box setting (i.e., model architecture and parameters, and datasets)
à impossible and unrealistic (privacy and security)
§ Black-box setting
Ø Reinforcement Learning (RL) -- Query Feedback (Reward)
Challenges
22
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

23. 25
CopyAttack
1
3
6
2
7
4
Target RecSys in Target Domain A
User Profile Selection in Source Domain B
User Profile Crafting
in Source Domain B
Copying User Profile
(Injection Attack)
Reward (+/-)
Users:
Items:
Reward (+/-)
(+) (-)
Top-k List
Feedback
(Queries)
5
PN-1
PN-2
PN-3
Start Point
PN-Length
End Point
Target Item to
be Attacked
Path for
User Profile Selection
Mask
(Stop Sign)
Raw
User Profile
Crafting
User Profile
Spy Users Real Users
No.
Non-leaf
Node
Users with
Target Item
Users without
Target Item
Sharing Items
in Two Domains
PN-*
Policy Network
Target Item
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

24. 26
User Profile Selection
• User Profile Selection
• Construct hierarchical clustering tree
• Masking Mechanism - specific target items
• Hierarchical-structure Policy Gradient
Time Complexity：
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

25. • User Profile Crafting
• Clipping operation to craft the raw user profiles
• Sequential patterns (forward/backward)
27
User Profile Crafting
w = 50%
Example:
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.

26. 28
CopyAttack
1
3
6
2
7
4
Target RecSys in Target Domain A
User Profile Selection in Source Domain B
User Profile Crafting
in Source Domain B
Copying User Profile
(Injection Attack)
Reward (+/-)
Users:
Items:
Reward (+/-)
(+) (-)
Top-k List
Feedback
(Queries)
5
PN-1
PN-2
PN-3
Start Point
PN-Length
End Point
Target Item to
be Attacked
Path for
User Profile Selection
Mask
(Stop Sign)
Raw
User Profile
Crafting
User Profile
Spy Users Real Users
No.
Non-leaf
Node
Users with
Target Item
Users without
Target Item
Sharing Items
in Two Domains
PN-*
Policy Network
Target Item
Attacking Black-box Recommendations via Copying Cross-domain User Profiles, ICDE, 2021.