COLLABORATIVE
FILTERING USING
ORTHOGONAL
NONNEGATIVE MATRIX
Presenter: Meng-Lun Wu
Authors: Gang Chen, Fei Wang and Changs...
OUTLINE
 Introduction
 Related Work
 Orthogonal nonnegative matrix tri-factorization
 Framework
 Experiments
 Conclu...
INTRODUCTION
 Collaborative filtering can predict a test user’s
rating for new items based on similar users.
 Collaborat...
INTRODUCTION (CONT.)
 This paper apply orthogonal nonnegative matrix
tri-factorization(ONMTF) to circumvent the two
kinds...
RELATED WORK
 Researchers have proposed some hybrid
approaches in order to combine the memory
based and model based appro...
RELATED WORK (CONT.)
 Matrix decomposition can used to solve the co-
clustering problem.
 Ding et al. (2005) proposed co...
ORTHOGONAL NONNEGATIVE
MATRIX TRI-FACTORIZATION
 The NMF is first brought into machine learning
and data mining fields by...
ORTHOGONAL NONNEGATIVE
MATRIX TRI-FACTORIZATION
(CONT.)
 The optimization problem can be solved using the
following updat...
FRAMEWORK
 Notations
 X = [u1,…,up]T
, uj=(xj1,…,xjn)T
, j∈{1,…,p}
 X = [i1,…,in], im=(x1m,…,xpm)T
, m∈{1,…,n}
9
MEMORY-BASED APPROACHES
 User’s neighbor selection
 Compute the similarities between a user and all the
user-cluster cen...
MEMORY-BASED APPROACHES
 The adjusted-cosine similarity between the m1th
and m2th items. (T is the set of users who both
...
ALGORITHM
1. The user-item matrix X is factorized as USVT
by using
ONMTF.
2. Calculate the similarities between the test u...
EXPERIMENTS
 Dataset
 MovieLens: 500 users and 1000 items (1-5 scales)
 Training set: the first 100, 200 and 300 users,...
DIFFERENT CLUSTER
14
 The ML_300 dataset is used for training, and try
10 different values of k or l (2,5,10,20,…,80)
PERCENTAGE OF NEIGHBORS
 The percentage of pre-selected neighbors reaches
around 30%.
15
SIZE OF NEIGHBORS
16
COMBINATION COEFFICIENTS
 Fix λ=0, the optimal value of δ is approximately
between 0.5 and 0.7.
17
COMBINATION COEFFICIENTS
(CONT.)
 Fix δ=0.6, the optimal value of λ is approximately
between 0.2 and 0.4.
18
PERFORMANCE COMPARISON
19
 Wang et al., 2006, similarity fusion (SF2)
 Xue et al., 2005, cluster-based Pearson correlati...
CONCLUSIONS
 This paper presented a novel fusion framework
for collaborative filtering.
 The model-based and memory-base...
Upcoming SlideShare
Loading in …5
×

Collaborative filtering using orthogonal nonnegative matrix

1,936 views
1,824 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,936
On SlideShare
0
From Embeds
0
Number of Embeds
318
Actions
Shares
0
Downloads
30
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Collaborative filtering using orthogonal nonnegative matrix

  1. 1. COLLABORATIVE FILTERING USING ORTHOGONAL NONNEGATIVE MATRIX Presenter: Meng-Lun Wu Authors: Gang Chen, Fei Wang and Changshui Zhang Source: Information Processing and Management (2009), pp. 368-379 1
  2. 2. OUTLINE  Introduction  Related Work  Orthogonal nonnegative matrix tri-factorization  Framework  Experiments  Conclusion 2
  3. 3. INTRODUCTION  Collaborative filtering can predict a test user’s rating for new items based on similar users.  Collaborative filtering can be categorized into…  Memory-based (similarity)  user-based and item-based  Model-based  Establish a model using training examples. 3
  4. 4. INTRODUCTION (CONT.)  This paper apply orthogonal nonnegative matrix tri-factorization(ONMTF) to circumvent the two kinds of collaborative filtering.  ONMTF is applied to simultaneously co-cluster the rows and columns, and attain individual predictions for an unknown test rating.  This paper possesses the following superiorities:  Sparsity problem  Scalability problem  Fusing prediction results 4
  5. 5. RELATED WORK  Researchers have proposed some hybrid approaches in order to combine the memory based and model based approaches.  Xue et. al. (2005) resolve the sparsity and scalability by using clusters to smooth ratings and clustering.  However, Xue only consider user-based approach, this paper extend the idea to integrate model based, user-based and item-based approaches. 5
  6. 6. RELATED WORK (CONT.)  Matrix decomposition can used to solve the co- clustering problem.  Ding et al. (2005) proposed co-clustering based on nonnegative matrix factorization. (NMF)  In 2006, they proposed ONMTF.  Long et al. (2005) provided co-clustering by block value decomposition. 6
  7. 7. ORTHOGONAL NONNEGATIVE MATRIX TRI-FACTORIZATION  The NMF is first brought into machine learning and data mining fields by Lee et al. (2001).  Ding et al. (2006) proved the equivalence between NMF and K-means, and extended NMF to ONMTF.  The idea is to approximate the original matrix X to the combination matrix, and the optimization problem is 7 lnlkkpnp TTT VSU andwhere IVVIUUtsUSVX × + × + × + × + ≥≥≥ ℜ∈ℜ∈ℜ∈ℜ∈ ==− VS,U,X ,..,min 2 0,0,0
  8. 8. ORTHOGONAL NONNEGATIVE MATRIX TRI-FACTORIZATION (CONT.)  The optimization problem can be solved using the following update rules.  After co-clustering, we could get the user centroid SVT and item centroid US. 8 ( ) ( ) ( )ik TT ik T ikik ik TT ik T ikik jk TT jk T jkjk VUSVU XVU SS XVSUU XVS UU USXVV USX VV )( )( )( ← ← ←
  9. 9. FRAMEWORK  Notations  X = [u1,…,up]T , uj=(xj1,…,xjn)T , j∈{1,…,p}  X = [i1,…,in], im=(x1m,…,xpm)T , m∈{1,…,n} 9
  10. 10. MEMORY-BASED APPROACHES  User’s neighbor selection  Compute the similarities between a user and all the user-cluster centroids SVT .  Select the top K user cluster as the user set uh.  The item’s neighbor selection is similar.  The cosine similarity between the j1th user and the j2th user.  Given an user-item pair <uj, im>, where uh∈{the most similar K-user of uj}. 10 ∑∑ ∑ == = = n m mj n m mj n m mjmj xx xx sim 1 2 1 2 1 )()( ))(( ),( 21 21 21 jj uu ∑ ∑ − += h h u jh u jh uu uu ),( ))(,( sim uusim ux hhm jjm
  11. 11. MEMORY-BASED APPROACHES  The adjusted-cosine similarity between the m1th and m2th items. (T is the set of users who both rated m1 and m2)  Given an user-item pair <uj, im>, where ih∈{the most similar K-items of im}  The final prediction result could be linearly combined the three different types of predictions. 11 ∑∑ ∑ ∈∈ ∈ −− −− = Tt ttmTt ttm Tt ttmttm uxux uxux sim 22 )()( ))(( ),( 21 21 21 mm ii ∑ ∑= h h i mh i mh ii ii ),( ))(,( sim xsim x jh jm jmjmjmjm xixuxnx ~~ )1)(1(~~)1(~~~ λδλδλ −−+−+=
  12. 12. ALGORITHM 1. The user-item matrix X is factorized as USVT by using ONMTF. 2. Calculate the similarities between the test user/item and user/item-cluster centroids. 3. Sort the similarities and select the most similar C user/item clusters as the test user/item neighbor candidate set. 4. Identify the most K neighbors of the test user/item by searching for the user/item candidate set. 5. Predict the unknown ratings by using user based and item based approaches. 6. Linearly combine three different predictions. 12
  13. 13. EXPERIMENTS  Dataset  MovieLens: 500 users and 1000 items (1-5 scales)  Training set: the first 100, 200 and 300 users, called ML_100, ML_200 and ML_300.  Testing set: the last 200 users  We randomly selected 5, 10 and 20 items rated by test users, called Given5, Given10 and Given20.  Evaluation metric  Mean absolute error (MAE) as evaluation metric.  Where N is the number of tested ratings. 13 N xx MAE mj jmjm∑ − = , ~
  14. 14. DIFFERENT CLUSTER 14  The ML_300 dataset is used for training, and try 10 different values of k or l (2,5,10,20,…,80)
  15. 15. PERCENTAGE OF NEIGHBORS  The percentage of pre-selected neighbors reaches around 30%. 15
  16. 16. SIZE OF NEIGHBORS 16
  17. 17. COMBINATION COEFFICIENTS  Fix λ=0, the optimal value of δ is approximately between 0.5 and 0.7. 17
  18. 18. COMBINATION COEFFICIENTS (CONT.)  Fix δ=0.6, the optimal value of λ is approximately between 0.2 and 0.4. 18
  19. 19. PERFORMANCE COMPARISON 19  Wang et al., 2006, similarity fusion (SF2)  Xue et al., 2005, cluster-based Pearson correlation coefficient (SCBPCC)  Rennie and Srebro, 2005, maximum margin matrix factorization (MMMF)  Ungar and Foster, 1999, cluster-based collaborative filtering (CBCF)  Hofmann and Puzicha, 1999, aspect model (AM)  Pennock et al., 2000, personality diagnosis (PD)  Breese et al., 1998, user-based Pearson correlation coefficient (PCC)
  20. 20. CONCLUSIONS  This paper presented a novel fusion framework for collaborative filtering.  The model-based and memory-based and naturally assembled via ONMTF.  Empirical studies verified our framework effectively improves the prediction accuracy.  Future work is investigate new co-clustering techniques and develop better fusion models. 20

×