The document discusses matrix factorization techniques for collaborative filtering recommender systems. It explains that matrix factorization involves decomposing a user-item rating matrix into the dot product of two lower-rank matrices containing latent user and item factors. This technique can be used to predict missing ratings and make recommendations. The document also covers evaluating recommender system performance and visualizing the learned latent factors.

1. RRecommend Roadtrip – Matrix Factorization
DYLAN VALERIO
Recommendation
Systems using R
WEAR SEATBELTS

2. 75% of watched series came from the recommender systems. (McKinsey)
35% of revenue comes from recommenders (McKinsey)
135.99 billion in revenue (Statista)

3. DYLAN VALERIO – NOVEMBER 2020
Dylan Valerio
•Software engineer
•Data Scientist
•Kaggler
•academic
•ADMU, BS CS,
•UP Diliman, MS CS

4. DYLAN VALERIO – NOVEMBER 2020
Review: Different Paradigms of Recommendation
watched
Content-based Filtering Collaborative Filtering
watched
Similar tags: crime, Robert de Niro, dark, mob
recommends
recommends
Pros More interesting for users
Cons Items with no usage (Cold-start)
Pros Readily explainable; Fast
Cons Stale and unchanging
• No free lunch
• It’s a quickly growing field with vast literature and domain-specific nuances

5. DYLAN VALERIO – NOVEMBER 2020
Review: Different Paradigms of Recommendation
Recommenders
Content-Based Filtering Collaborative Filtering
Neighborhood-based
(Memory-based)
Model-based
(Vector-space models)
Hybrid

6. DYLAN VALERIO – NOVEMBER 2020
Review: Memory-Based Models
User-based K-Nearest Neighbor Recommendation
Intuition: Find the most enjoyed items by the users closest to me in terms of what they watch.
Item-based K-Nearest Neighbor Recommendation
Intuition: Find the closest items of the items I enjoyed in terms of the users that enjoyed both.
Similarity between users
Similarity between items

7. DYLAN VALERIO – NOVEMBER 2020
Anime Recommendations

8. DYLAN VALERIO – NOVEMBER 2020
Ratings Matrix
Conceptual View Data View

9. Collaborative Filtering – Matrix Factorization
IF YOU CAN’T EXPLAIN IT SIMPLY, YOU DON’T UNDERSTAND IT WELL ENO UGH

10. DYLAN VALERIO – NOVEMBER 2020
Factorization
• 32 and 27 has special properties
• Tons of algorithms for factorization
• P & Q produces R
• In some algorithms, P & Q have
special properties

11. DYLAN VALERIO – NOVEMBER 2020
Intuitions
If you can imagine all movies in 2 dimensions:
https://developers.google.com/machine-learning/crash-course/embeddings/motivation-from-collaborative-filtering Microsoft blog (source image URL missing)
Imagine also users in the same space

12. DYLAN VALERIO – NOVEMBER 2020
Other Areas of Application
https://www.researchgate.net/figure/The-approaches-
based-on-a-low-rank-assumption-Non-negative-matrix-
factorization-NMF-is_fig2_324487531
An Overview of Lead and
Accompaniment Separation in Music
EigenFaces and A Simple
Face Detector
https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-
a-simple-face-detector-with-pca-svd-in-python/
Low Rank plus Sparse Matrix recovery using
Randomized Rank Revealing Decomposition
https://www.youtube.com/watch?v=70DMDJUxyl8

13. DYLAN VALERIO – NOVEMBER 2020
Matrix Factorization For Recommenders
• Latent factors describe the structure of the data
beyond the noise
• There are two latent variables, the items and the
users, rows and columns, respectively.
• Can “recover” the missing values in the ratings matrix
Similarity is defined as the dot product
This computes for R even for content not consumed by user i.
User
Item
• Algorithms for MF: Alternating Least Squares,
FunkSVD, Weighted Matrix Factorization
• Minimize the following loss function:

14. DYLAN VALERIO – NOVEMBER 2020
RecommenderlabRecommendation library in R
Convert data table to matrix of users in rows and items in columns
Input recommender algorithm
1
2
https://www.kaggle.com/krsnewwave/r-study-factorization

15. Evaluating Recommendation Algorithms
WHO’S A GOOD BOY?

16. DYLAN VALERIO – NOVEMBER 2020
Evaluating a Good Recommender
We take out a fraction of watches from each user.
We then compare the similarity of our predicted
recommendations to actual watched items.
Ranking Metrics
• Precision
• Recall
• Area under the ROC curve
Error-Based Metrics
• RMSE, MAE
Other metrics
• Diversity
• Novelty
• Serendipity

17. DYLAN VALERIO – NOVEMBER 2020
Recommenderlabs Evaluation
Create evaluation strategy
1 2
3
Evaluate top N of the list (1, 3, 5… items)
Competing algorithms:
SVD – 10 traits (k)

18. DYLAN VALERIO – NOVEMBER 2020
Evaluation
Results

19. DYLAN VALERIO – NOVEMBER 2020
Evaluation Results
Some manipulation of the resulting
table and ggplot magic.

20. DYLAN VALERIO – NOVEMBER 2020
Evaluation Results
Some manipulation of the resulting
table and ggplot magic.

21. Interpreting the Output Matrices
LOGIC WILL GET YOU FROM A TO B. IMAGINATION WILL TAKE YOU EVERYW HERE.

22. DYLAN VALERIO – NOVEMBER 2020
Interpretation / Visualization TechniquesUsing factoextra
Idea: Run Principal Components Analysis for
Factor Analysis (2-stage decomposition)
Each row in P and Q are user and item
features
Import Libraries for Factor Analysis
Get Item Latent Vectors
1
2

23. DYLAN VALERIO – NOVEMBER 2020
Principal Components’ Contribution
• 1st component accounts
for 12% of variance
• The next 8 accounts for
80%
• The 10th is around 7.5%
• After the 10th, there are
only minor contributions.
1 5 10

24. DYLAN VALERIO – NOVEMBER 2020
Visualizing Individual Shows
Similarly rated shows are clustered
together

25. DYLAN VALERIO – NOVEMBER 2020
Visualizing Individual Shows and the Original Features
V1 has a small angle with first
principal component

26. DYLAN VALERIO – NOVEMBER 2020
itstherealdyl.com
Kaggle Data: Blood Chemistry Kaggle Competition: Melanoma A Name Across Time and Space

27. DYLAN VALERIO – NOVEMBER 2020
RRecommend Roadtrip
DYLAN VALERIO
Recommendation
Systems using R
WEAR SEATBELTS