Recommender System are backbone of e-commerce websites. But now a days we can not relay 100% on the performance of RS. Attackers can affect the results of recommender system. It become very important for service provider to reduce the impact of attacks.

1. Detection of Profile Injection Attacks
in
Recommender System
Presented By:
Ashish Pannu

2. Agenda
• Recommender System
- Introduction
- Types of Recommender System
- Why using Recommender System
• Profile Injection Attacks
- Why Attacker Attacks on Recommender System (Example)
- Attack Profile Structure
- Types of Attacks
o Push Attacks
o Nuke Attacks
- Attack Detection Attributes
o Generic Attributes
o Model Specific Attributes
• Proposed Approach
• Conclusion
• References
2

3. Recommender System
Introduction:
– Recommender System (RS) is based on information filter to predict
the rating or preference that a user would give to an unseen item [1].
– It is backbone of ecommernce websites (amazon, flipkart, myntra
etc.), social networking websites (facebook, linkedin, twitter, google +
etc.), matrimonial websites (shaadi.com, bharatmatrimonial.com etc.)
and many more.
3

4. Types of RS-
1. Collaborative Filtering (CF) based RS: based on correlation between
different users. It states that if two user had similar tastes in past will also
have same taste in future also.
– Most popular and widely used.
2. Content based RS: based on the information on the features(keywords)
of items rather than on the opinion of other users. E.g. in movie
recommendation keywords may be: movie name, actor, actress, genre
etc.
3. Knowledge based RS: customer defines his requirement explicitly. E.g. “I
want black color BMW car”.
Recommender System
4

5. Recommender System – why?
1. Value for the customer
– Narrow down the set of choice.
– Discover new things.
– Just exploring new items.
– Make shopping easier.
2. Value for the provider
– Increase user satisfaction.
– Increase the sell.
– Unique and personalized service to each customer.
– Obtain more knowledge about the customer.
– Opportunities for promotions.
5

6. Profile Injection Attack
Profile injection attack: Example
• Assume that user-user based CF is used.
• Pearson correlation as similarity measure.
• Neighborhood size of 1.
Item1 Item2 Item3 Item4 … Target Pearson
Ram 5 3 4 1 … ?
User1 3 1 2 5 … 5 -0.54
User2 4 3 3 3 … 2 0.68
User3 3 3 1 5 … 4 -0.72
User4 1 5 5 2 … 1 -0.02
6

7. Profile Injection Attack
Profile injection attack: Example
• Assume that user-user based CF is used.
• Pearson correlation as similarity measure.
• Neighborhood size of 1.
Item1 Item2 Item3 Item4 … Target Pearson
Ram 5 3 4 1 … ?
User1 3 1 2 5 … 5 -0.54
User2 4 3 3 3 … 2 0.68
User3 3 3 1 5 … 4 -0.72
User4 1 5 5 2 … 1 -0.02
User2 most similar to Ram
7

8. Profile Injection Attack
Profile injection attack: Example
• Assume that user-user based CF is used.
• Pearson correlation as similarity measure.
• Neighborhood size of 1.
User2 most similar to Ram
Item1 Item2 Item3 Item4 … Target Pearson
Ram 5 3 4 1 … ?
User1 3 1 2 5 … 5 -0.54
User2 4 3 3 3 … 2 0.68
User3 3 3 1 5 … 4 -0.72
User4 1 5 5 2 … 1 -0.02
Attack 5 3 4 3 … 5 0.87
Attack
8

9. Profile Injection Attack
Profile injection attack: Example
• Assume that user-user based CF is used.
• Pearson correlation as similarity measure.
• Neighborhood size of 1.
Item1 Item2 Item3 Item4 … Target Pearson
Ram 5 3 4 1 … ?
User1 3 1 2 5 … 5 -0.54
User2 4 3 3 3 … 2 0.68
User3 3 3 1 5 … 4 -0.72
User4 1 5 5 2 … 1 -0.02
Attack 5 3 4 3 … 5 0.87
Attack
Attack most similar to Ram
9

10. Attack Profile Structure
In order to look similar to genuine user, attacker gives ratings in a specific
manner so that it become hard to identify the attack profile [2].
Selected items: these items are chosen because of their association with the
target item.
Filler items: these items are randomly chosen and rating is given based upon
properties of attack.
Unrated items: No ratings are given to these items.
Target item: singleton item to which attacker promote or demote.
Item1 … ItemK … ItemN … ItemR Target
r_1 … r_k … r_l … r_n X
Selected items Filler items Unrated items Target item
10

11. Types of Attacks
Push Attacks- to promote a specific item. Maximum rating will be given to
the target item [3].
1. Random Attack:
– No rating to selected items.
– Average rating of system to the filler items.
– Least cost attack.
– Limited effect as compared to more advance attacks.
2. Average Attack:
– No rating to selected items.
– Average rating of item to the item of filler items.
– Additional cost of finding the average rating is involved.
– More effective as compared to random attack.
cont …
11

12. Types of Attacks
3. Bandwagon Attack:
– Maximum rating is given to popular items (selected items).
– Average rating of system to the filler items.
– It is a low cost attack.
4. Segment Attack:
– Maximum rating is given to the items of the same segment of target
item (selected items).
– Minimum rating is given to filler items.
– Best attack (impact wise) as compared to other attacks of same
category.
cont …
12

13. Types of Attacks
Nuke Attack- to demotion of an item. Minimum rating is given to the target
item.
1. Love/Hate Attack:
– No rating to selected items.
– Maximum rating to set of filler items.
2. Reverse Bandwagon Attack:
– Minimum rating is given to least popular items (selected item).
– Average rating of system is given to the set of filler items.
– More effective as compared to the love/hate attack.
We can not stop attacks, we can just increase the cost of attacks.
13

14. Attack Detection Attributes
Generic Attributes- based on general abnormal behavior of user [4]. It is
common for all attack types.
1. Rating Deviation from Mean Agreement (RDMA): finds profile’s average
rating deviation per item.
is the number of ratings given by user u. is rating given by user u
to item i. is average rating of item i. is number of ratings provided
for item i.
2. Degree Similarity with Top Neighbors (DegSim): find the average
similarity of profile with top k neighbors.
Nu r i,u
ri ti
14

15. Attack Detection Attributes
3. Length Variance: finds the variance in the length (# ratings given) of a given
profile from the average length of database.
Model Specific Attributes- focuses on the signature of attacks type.
1. Mean Variance (MeanVar): used for average and random attack.
is the set of items rated by user u.
2. Filler Mean Target Difference (FMTD): used for bandwagon, reverse
bandwagon and segment attack.
is the set of filler items rated by user u.
pu
p f,u
15

16. Evaluation Metrics
1. Precision: count total number of profiles that are labeled as attack [4].
2. Recall: count total number of actual attack in the system.
Reality
Prediction
Actually Attack Actually Good
Rated Attack True Positive (Tp) False Positive (Fp)
Rated Good False Negative (Fn) True Negative (Tn)
16

17. Proposed Approach
Steps-
1. Calculate the attack detection attributes (generic and model specific
attributes).
2. Apply statistical models using k-fold cross validation
3. Compare the accuracy of statistical models.
4. Pick the top three performing models.
5. Ensemble the models (resulting models from step 4) using voting
approach.
Data Set Used: MovieLens-100K
– 943 users
– 1682 movies
– 100000 ratings
17

18. Performance Analysis
Table 1: Performance analysis of 10% average attack.
Table 2: Performance analysis for bandwagon attack at 5% filler size.
Filler Size 1% 10% 20% 30% 40% 50%
Models P R P R P R P R P R P R
Decision Tree .90 .892 .921 .93 .928 .919 .939 .912 .94 .94 .961 .968
Random Forest .929 .930 .939 .92 .948 .948 .952 .956 .962 .961 .973 .979
Ada Boost .9 .908 .914 .918 .93 .924 .935 .938 .948 .934 .950 .943
SVM .938 .927 .949 .94 .959 .959 .971 .975 .979 .981 .988 .989
Linear Regression .89 .862 .89 .907 .918 .91 .925 .918 .929 .902 .93 .931
Neural Network .949 .938 .95 .943 .951 .954 .959 .958 .968 .967 .979 .98
Ensemble .939 .932 .946 .934 .953 .953 .961 .963 .968 .969 .98 .982
Attack Size 1% 3% 6% 9% 12% 15%
Models P R P R P R P R P R P R
Decision Tree .822 .811 .844 .830 .859 .848 .868 .855 .889 .872 .901 .928
Random Forest .880 .872 .894 .897 .905 .912 .909 .902 .929 .917 .957 .922
Ada Boost .819 .802 .829 .810 .834 .828 .858 .840 .883 .868 .902 .915
SVM .901 .902 .912 .928 .928 .908 .937 .911 .972 .959 .99 .984
Linear Regression .862 .842 .872 .882 .882 .908 .908 .919 .918 .935 .939 .941
Neural Network .892 .901 .919 .911 .922 .929 .959 .958 .961 .969 .97 .964
Ensemble .891 .892 .908 .912 .918 .916 .935 .924 .954 .948 .972 .957
18

19. Conclusion
1. I tried to explain why RS is so important now a days for ecommerce.
2. I also tried to explain, how a attacker can manipulate the results of RS.
3. I focused on several keys areas of attacks against recommender system
i.e. different types of attacks, attack detection attributes and model
evaluation metrics.
4. I found that random forest, SVM and neural networks perform better
than other models in attack detection.
5. I present ensemble approach for the attack detection and I found that
performance is not best in any case but it also does not give worst
performance in any case.
19

20. 20

21. References
[1]. Davoodi, Fatemeh Ghiyafeh, and Omid Fatemi. "Tag based recommender
system for social bookmarking sites." In Advances in Social Networks
Analysis and Mining (ASONAM), 2012 IEEE/ACM International
Conference on, pp. 934-940. IEEE, 2012.
[2]. Lam, Shyong K., and John Riedl. "Shilling recommender systems for fun
and profit." In Proceedings of the 13th international conference on
World Wide Web, pp. 393-402. ACM, 2004.
[3]. Mobasher, Bamshad, Robin Burke, Runa Bhaumik, and Chad Williams.
"Toward trustworthy recommender systems: An analysis of attack
models and algorithm robustness." ACM Transactions on Internet
Technology (TOIT) 7, no. 4 (2007): 23.
[4]. O'Mahony, Michael, Neil Hurley, Nicholas Kushmerick, and Guénolé
Silvestre. "Collaborative recommendation: A robustness analysis." ACM
Transactions on Internet Technology (TOIT) 4, no. 4 (2004): 344-377.
21