This deck will discuss application of Matrix Factorization in Machine Learning. It will discuss Least Square Matrix Factorization, Poisson Matrix Factorization.

1. Matrix Factorization Algorithms
in Apache SystemML
Prithviraj Sen

2. Applications of Matrix Factorization
• Netflix Prize
• Given ratings data predict what movies
users will watch
1 3 4 ? ? ?
? 3 5 ? ? 5
? ? 4 5 ? 5
? ? 3 ? ? ?
? ? 3 ? ? ?
2 ? ? 2 ? 2
? ? ? ? 5 ?
? 2 1 ? ? 1
? 3 ? ? 3 ?
1 ? ? ? ? ?
17,700 movies
480,000
users

3. Applications: Parts Based Decomposition

4. Least Squares Matrix Factorization
• Approximate V using LR
min∑ (𝑣$%−𝑙$′𝑟%)2
($,%)
• Leads to the very well
known alternating least
squares algorithm
• Only requires solving least
squares
• Embarassingly parallel
V L
R
f
f
~~
x

5. ALS in DML
parfor
parfor
Direct solving
least squares
Direct solving
least squares

6. Poisson Matrix Factorization (NMF)
• Suitable if you are looking for non-negative factors
𝑣 = 𝑒/01
𝑙𝑟 𝑛/𝑛!
• Leads to the well known Generalized KL-Divergence
max∑ (𝑛𝑖𝑗log𝑙𝑖′𝑟𝑗 − 𝑙𝑖′𝑟𝑗)$%
• Well known update equations exist*
* “Generalized Nonnegative Matrix Approximations with Bregman Divergences” by Dhillon and Sra in NIPS 2005.
* “Distributed Nonnegative Matrix Factorization for Web-Scale Dyadic Data Analysis on MapReduce” by Liu et al in WWW 2010.

7. PNMF in DML
• Very efficient updates using only linear algebra
• Uses Apache SystemML’s wdivmm operator
wdivmm operator