Download to read offline
Uploaded byDataconomy Media
DN 2017 | Building a recommender system using collaborative filtering (CF) | Sarah Mestiri
This document discusses building a recommender system using collaborative filtering. It begins with an introduction to collaborative filtering and common methods like memory-based and model-based collaborative filtering. It then explains the process for memory-based collaborative filtering including similarity calculation, determining peer groups, and making recommendations. Model-based collaborative filtering is introduced using matrix factorization to predict ratings. The document concludes with the steps to build a recommender system which includes understanding data, pre-processing, building the collaborative filtering model, training and testing the model, and evaluating accuracy.
More Related Content
Similar to DN 2017 | Building a recommender system using collaborative filtering (CF) | Sarah Mestiri
![[Final]collaborative filtering and recommender systems](https://cdn.slidesharecdn.com/ss_thumbnails/finalcollaborativefilteringandrecommendersystems-141103044224-conversion-gate01-thumbnail.jpg?width=640&height=640&fit=bounds)
DN 2017 | Building a recommender system using collaborative filtering (CF) | Sarah Mestiri
- 1. Building a recommender systemusing Collaborative Filtering (CF) Sarah Mestiri Machine Learning Engineer
- 2. “Customer expect tobe treated as human, not number.” Introduction “State of the connected customer” report- Salesforce - 2016
- 3. Outline Experiencing personalization through collaborativefiltering Common-Methods in CF Building a recommender system using CF
- 4.
- 5.
- 6.
- 7.
- 8. Memory-based CF Types User-based collaborativefiltering Item-based collaborative filtering Recommendations based on similar users ratings to target user. Recommendations based on target user’s own ratings on similar items. Item-based
- 9. Memory-based CF Process explained Prediction: 1-Similarity calculation between items (users). 2-”Peer Group”: top-k most similar items (users). i1 i2 i3 i4 i5 u1 3 5 1 1 ? u2 1 0 0 2 1 u3 2 5 1 ? ? u4 0 1 1 ? ?
- 10. Memory-based CF Implementation & challenges Step1: Choice of Similarity Measure - Pearson (mean-centered ratings) - Cosine - Adjusted Cosine Step 2: Determination of Peer Group - Top-k most similar items (users) What if one user has a general tendency to rate generously while the other is harsh in his ratings? Step 3: Recommendation
- 11. Model-based CF Introduction Where doNeighborhood methods fail? Computation Scalability Sparsity Accuracy Challenges
- 12. What to recommendto Alice? Model-based CF
- 13. Matrix Inception FrozenKing-kong Zootopia Alice -1 -1 1 1 ? Patrick 1 0 0 -1 1 John 1 -1 -1 ? ? Sara 0 1 1 ? ? Matrix Inception Frozen King-kong Zootopia Children -1 -1 1 1 1 Action 1 1 -1 -1 0 Children Action Alice 1 -1 Patrick -1 1 John -1 1 Sara 1 0 Model-based CF Matrix factorization
- 14. X Matrix Inception FrozenKing-kong Zootopia Alice -1 -1 1 1 ? Patrick 1 0 0 -1 1 John 1 -1 -1 ? ? Sara 0 1 1 ? ? Matrix Inception Frozen King-kong Zootopia Children -1 -1 1 1 1 Action 1 1 -1 -1 0 Children Action Alice 1 -1 Patrick -1 1 John -1 1 Sara 1 0 Latent factors Model-based CF Matrix factorization
- 15. Model-based CF Ratings prediction Howcan one determine the factor matrices U and V ? Goal: Minimize the difference between predicted ratings & observed ratings. Stochastic Gradient Descent SGD (~SVD) or Alternating Least Squares ALS
- 16.
- 17. Step 1: Understandyour Data Visualizations (Pandas, Matplotlib, Seaborn)
- 18. Step 2: Pre-Processyour data Cleanup, merge, etc. (Pandas, Numpy)
- 19. Step 3: Buildyour ML Model Implement the CF method (KNN,SGD,ALS) or use it from available libraries (Scikit-learn, Surprise, MLLib Spark)
- 20. Step 4: Trainyour model Predict ratings on the train set.
- 21. Step 5: Testyour model Predict ratings on the test set.
- 22. Step 6: Evaluateyour model: Measure accuracy using RMSE
- 23.
- 24.
Editor's Notes
- #9 Advantages of item-based: leverage the user’s own ratings => more consistency. More stable with changes to the ratings. (adding new item VS adding new users : happening less VS more)
- #11 Similarity challenges: User Bias + Number of common ratings between users Peer Group: weakly or negatively correlated users + Long tail impact.
- #12 Compute all possible rating predictions for the relevant user-item pairs (e.g all items for a particular user) and then rank them. => Computationally expensive / possible MemoryError
- #15 Exploit the fact that significant portions of the rows and columns of data matrices are highly correlated. Data redundancies + high correlation => Low-rank Matrix.
- #16 State the advantages of ALS