Joonyoung Yi, profile picture
Uploaded byJoonyoung Yi
PDF, PPTX1,837 views

Why biased matrix factorization works well?

The document discusses the Netflix Prize competition to improve movie recommendation accuracy, where the winning algorithm was based on biased matrix factorization. It provides background on the Netflix data and evaluation metrics used. Matrix factorization techniques like Funk SVD, probabilistic matrix factorization, and alternating minimization are introduced as important algorithms for collaborative filtering and recommendation systems.

Embed presentation

Download as PDF, PPTX
Why Biased Matrix Factorization Works Well? 2018. 8. 29. JoonyoungYi joonyoung.yi@kaist.ac.kr
 Machine Learning & Intelligence Laboratory School of Computing
 Korea Advanced Institute of Science andTechnology
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
NETFLIX PRIZE • In October 2016, Netflix open a $1M Prize called Netflix Prize. • To improve recommendation accuracy by 10%. • They splitted the data for prize. • Training set: 100M • Test set: 1.4M • Training set contains 1.2 % of entries. • 480,000 users and 17,700 movies. • From a matrix viewpoint, training set contains only 1.2% entries of the matrix. !4 1 2 3 4 5 6 7 8 … 1 4 2 4 2 3 3 3 3 3 1 4 2 4 1 5 … 2 3 1 Users Movies 480,000 17,700
DATA DESCRIPTION OF NETFLIX PRIZE • Each set(training set, test set) is a set of (i, j, Mij)s. • i: user index • j: item(or movie) index • Mij: true rating of user u to item i. • : the set of (i, j) corresponding to the training set(=The set of known entries). • : the set of (i, j) corresponding to the test set(=The set of entries to predict). • m is the number of users. • n is the number of items. !5 ⌦<latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit> ⌦0 <latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
DATA DESCRIPTION OF NETFLIX PRIZE • RMSE(Root Mean Square Error) was used for measuring accuracy. • Rij: predicted rating of user i to item j. • is the cardinality of the . • Projection Operator • M, R are rating matrices. • !6 ⌦0 <latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit> |⌦0 |<latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit> train RMSE = s X ⌦ (Mij Rij)2 |⌦| = kP⌦(M R)kF p ⌦ test RMSE = s X ⌦0 (Mij Rij)2 |⌦0| = kP⌦0 (M R)kF p ⌦0 <latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit> P⌦(M) = ⇢ Mij if (i, j) 2 ⌦, 0 otherwise.<latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit> M, R 2 Rm⇥n <latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit>
RMSE on Test Set RMSE OF NETFLIX, PRIZE WINNER AND BIASED-MF • RMSE of Cinematch (Original Netflix Recommendation Engine): 0.9514 • RMSE of Winner: 0.8553 (10% enhancement) • They ensembled a lot of algorithms. • RMSE of Biased-MF ONLY: 0.8799 (7.5% enhancement) • Some materials said 6% enhancement only by Biased-MF. • RMSE of Biased-MF(with tuned hyper-parameters): 0.844 (11.3% enhancement) • From the AutoRec Paper in WWW’15. !7
BIASED-MF IS FAST AND WORKS WELL • Many Algorithms are presented after Biased-MF. • Biased-MF is the fastest algorithm among them. • The convex relaxation(nuclear norm minimization) algorithm is slower than Biased-MF. • Biased-MF works quite well even these days. • And, the some of algorithms with good performance are based on Biased-MF. • SMA and MRMA are ensemble algorithms based on Biased-MF. • LLoRMA and ABCF are modification of Biased-MF. !8 Movielens 1M Movielens 10M Netflix(100M) Biased MF(IEEE’09) 0.845 0.803 0.844 LLoRMA(ICML’13) 0.8333 0.7815 0.8337 AutoRec(WWW’15) 0.831 0.782 0.823 CF-NADE(ICML ’16) 0.829 0.771 0.803 SMA(ICML’16) - 0.7682 0.8036 ABCF(Neurocomputing’18) 0.836 0.766 0.795 MRMA(NIPS’17) - 0.7634 0.7973
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
FUNK SVD(SINGULAR VECTOR DECOMPOSITION) • Before introducing Biased-MF, Let’s see Funk SVD(suggested by Funk). • Goal of Netflix Prize: Minimize RMSE • There is no way to solve this optimization form without any assumption. • In rating recommendation, it is natural to assume the matrix is low-rank. • New Optimization Form with Low-rank Assumption: • p and q are low-rank matrices: • To avoid overfitting: !10 ˆrui<latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit> arg min R v u u t X (i,j)2⌦ (Mij Rij)2 |⌦| = arg min R X (i,j)2⌦ (Mij Rij)2 <latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 <latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit> max(rank( ˆU), rank( ˆV )) ⌧ dim(M)<latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit>
SOLUTION OF THE FUNK SVD • Optimization form of the Funk SVD: • If we know all entries (= contains all (u, i) pairs), we can solve by SVD (Singular vector decomposition). • However, we can’t know the all entries. • If we don’t know all entries, this is NP-hard problem. • Then, how to solve this optimization problem? • One of approaches is convex relaxation. • But, the Funk-SVD use another algorithm. !11 ⌦<latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
SOLVE BY ALTERNATING MINIMIZATION !12 M ≒ xm m n n k k These can be solved by SVD! (pseudo inverse) ˆU<latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit> ˆV † <latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit> 1: Input: observed set ⌦, values P⌦(M) 2: Initialize ˆV 0 randomly. 3: for t = 1, · · · , T: 4: ˆUt arg min U2Rm⇥k kP⌦(M U( ˆV t 1 )† )k2 F + U kUk2 F 5: ˆV t arg min V 2Rk⇥n kP⌦(M ˆUt V † )k2 F + V kV k2 F 6: Return R = ˆUt ( ˆV t )† <latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit>
WHAT IS BIASED MATRIX FACTORIZATION? • The Biased Matrix Factorization is based on Funk SVD. • Biased MF = Funk SVD + Bias Terms • Some users tend to give higher ratings, and some users tend to give lower ratings. • The tendencies of movies are similar to that of users. • Some movies tend to get higher ratings, and some movies tend to get lower ratings. • Biased Matrix Factorization wants to handle this phenomenon. • To introduce bias terms in the optimization form. !13
OPTIMIZATION FORM OF BIASED-MF • Therefore, Biased-MF introduce biased terms related to user and item respectively. • is the average rating of training set(constant). • is the bias vector related to user. • is the bias vector related to item. • This optimization form can be solved by alternating minimization. • Similar to the solution of Funk SVD. • The Biased Matrix Factorization is also called SVD++. !14 µ<latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit> buser <latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit> bitem <latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit> min ˆU, ˆV ,buser,bitem X (i,j)2⌦ (Mij µ buser i bitem j ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F + buser kbuser k2 2 + bitem kbitem k2 2<latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit>
THE ROLE OF BIAS TERMS IN BIASED-MF • The bias terms serve the role of normalization. • The bias terms serve to make the rating matrix r be zero mean. • Meanwhile, the Biased-MF without bias terms(=Funk SVD) also works quite well. • The RMSE performance on Netflix Dataset is similar to original recommendation engine of Netflix. • Therefore, knowing why Biased-MF works well is equivalent to
 knowing why Funk SVD works well. • Now, we will investigate why Funk SVD works well on low-rank matrices. !15
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
OPTIMIZATION FORM OF FUNK SVD • Recall: Optimization form of Funk SVD • Probabilistic Matrix Factorization (PMF, NIPS’08) interprets Funk SVD 
 in the view point of posterior. • Funk SVD can be interpreted as an algorithm that finds the most probable low-rank matrices p and q when a star is observed. !17 arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
PROBABILISTIC MATRIX FACTORIZATION • Optimization form of PMF: • To find and via MAP(Maximum A Posteriori). • A low-rank assumption ( ) and Gaussian noise assumptions. !18 Mij i=1,…,m j=1,…,n σ σU σV arg max ˆU, ˆV Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] <latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit> ˆUi<latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit> ˆVj<latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit> Mij = ˆUi ˆV † j<latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit> Pr[M| ˆU, ˆV , 2 ] = mY i=1 nY j=1 [N[Mij| ˆUi ˆV † j , 2 ]]I⌦ ij , Pr[ ˆU| 2 U ] = mY i=1 N[ ˆUi|0, 2 U I], Pr[ ˆV | 2 V ] = nY j=1 N[ ˆVj|0, 2 V I], I⌦ ij = ⇢ 1 (i, j) 2 ⌦ 0 otherwise.<latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit> ̂U ̂V
THE LOG POSTERIORI OF PMF • The log posteriori of PMF is as follows: • Hence, the following equation holds: !19 arg max ˆU, ˆV Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] = arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit> ln Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] = 1 2 2 mX i=1 nX j=1 I⌦ ij(Mij ˆUi ˆV † j )2 1 2 2 U mX i=1 ˆU† i ˆUi 1 2 2 V nX j=1 ˆV † j ˆVj 1 2 (( mX i=1 nX j=1 I⌦ ij) ln 2 + mk ln 2 U + nk ln 2 V ) + C <latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit>
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
WHY WE USE ALTERNATING MINIMIZATION? • We can just use Gradient Descent with random initialization without alternating minimization. • Why we should use alternating minimization? • First of all, Gradient Descent can oscillate without alternating minimization. • The MRMA paper(NIPS’17) said that it is possible to overfit without alternating minimization. • Alternating Minimization is not the only way to avoid overfitting. • However, Besag(1986) said that it is a good way to avoid overfitting. • Glendinning(1989) said that alternating minimization is robust to initial point empirically. • However, the initial point is nonetheless important. • Because it can fall into the local minima. !21
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
LOCAL MINIMA PROBLEM OF AM • In otherwise, Jain et al(2013) showed that AM has no local minima problem. • Noiseless case ONLY. Exact Completion Setting. • With the slightly modified algorithm. • With some assumptions. • Let’s look at the modified algorithm first. !23
THE MODIFIED ALGORITHM !24
THE MODIFIED ALGORITHM !25 Mini-batch
THE MODIFIED ALGORITHM !26 Alternating Minimization similar to Funk SVD
THE MODIFIED ALGORITHM !27 When performing initialization using SVD + Clipping, the local optimum found by the SVD method
 is close enough to the global optimum!
MOTIVATION OF THE INCOHERENCE ASSUMPTION • Consider the rank-1 matrix M: • Let |Ω| be the number of observed entries of M. • Then, we can see only 0 with probability 1 - |Ω| / (mn). • If sample set doesn’t contain 1, we can not complete matrix exactly. • Therefore, it is impossible to recover all low-rank matrices. !28 M = e1e⇤ n = 2 6 6 6 4 0 0 · · · 0 1 0 0 · · · 0 0 ... ... ... ... ... 0 0 · · · 0 0 3 7 7 7 5 <latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit>
THE INCOHERENCE ASSUMPTION • More generally, it is hard to recover if the singular vectors of the matrix M are similar to standard basis. • Because, information is highly concentrated on specific region. • Hence, the singular vectors need to be sufficiently spread. • This is why the paper introduce the incoherence assumption. • The EMCCO paper said that the random low-rank (orthogonal) matrices satisfy the incoherent assumption. !29
MAIN RESULTS • Required entries: |Ω| = O((k4.5 log k) n log n) • Required steps: O(log (1/ε)) • The EMCCO paper said that the optimum number of required entries is O(n log n) !30
COMPARISON TO NUCLEAR NORM MINIMIZATION • Alternating Minimization • Required entries: |Ω| = O((k4.5 log k) n log n) • Required steps: O(log (1/ε)) • Nuclear Norm Minimization (Convex Relaxation) • Required entries: |Ω| = O((k) n log n) • Required steps: O(ε-1/2) • Nuclear Norm Minimization requires smaller the number of samples. • Alternating Minimization converges faster than Nuclear Norm Minimization. • Theoretical bound is not tight in Nuclear Norm Minimization. !31
PROOF SKETCH !32
PRELIMINARY !33
BASE CASE !34
INDUCTION STEP (DISTANCE) !35
INDUCTION STEP (INCOHERENCE) !36
TABLE OF CONTENTS 1. Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
WHY RANDOM INITIALIZATION WORKS? • In prior paper, initialization steps matters to guarantee global optimality. • In practice, it is hard to determine hyper-parameters for clipping. • However, random initialization works well practically. • In this section, I’ll show you to why random initialization works. • It can be view as removing hyper-parameters. • Similar to motivation of WGAN-GP. • WGAN is really hard to determine hyper-parameter that performance is highly sensitive to. !38
PRELIMINARY: MATRIX SENSING PROBLEM • Quite similar to the matrix completion problem. • A Low-rank version of linear regression (y = xβ problem). • RIP condition: • Actually, the Jain et al’s work (2013) proved in low-rank matrix sensing problem first. • And then, they extended their work to low-rank matrix completion problem. !39
NO SPURIOUS LOCAL MINIMA • Rong et al (NIPS 2016), Matrix Completion has No Spurious Local Minimum • Matrix Completion problem with Semi-definite matrix assumption. • Showed why random initialization works with proper regularizer. • Srinadh et al (NIPS 2016), Global Optimality of Local Search for Low Rank Matrix Recovery • Matrix Sensing Problem. • Showed all local minima are very close to a global optimum with noisy measurements. • With a curvature bound (RIP condition), 
 a polynomial time global convergence is guaranteed by SGD from random initialization. • Rong et al (ArXiv 2017), No Spurious Local Minima in Non-convex Low Rank Problems: 
 A Unified Geometric Analysis • Extended Srinadh et al’s works from matrix sensing to matrix completion and robust PCA. • Showed no high-order saddle point exists if being with proper regularizer. !40
ANY QUESTION?
REFERENCES [1] Koren,Yehuda, Robert Bell, and ChrisVolinsky. "Matrix factorization techniques for recommender systems." Computer 8 (2009): 30-37. [2] Mnih,Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances in neural information processing systems. 2008. [3] Jain, Prateek, Praneeth Netrapalli, and Sujay Sanghavi. "Low-rank matrix completion using alternating minimization." Proceedings of the forty-fifth annual ACM symposium on Theory of computing.ACM, 2013. [4] Ge, Rong, Jason D. Lee, and Tengyu Ma. "Matrix completion has no spurious local minimum." Advances in Neural Information Processing Systems. 2016. [5] Bhojanapalli, Srinadh, Behnam Neyshabur, and Nati Srebro. "Global optimality of local search for low rank matrix recovery." Advances in Neural Information Processing Systems. 2016. [6] Ge, Rong, Chi Jin, andYi Zheng. "No spurious local minima in nonconvex low rank problems:A unified geometric analysis." arXiv preprint arXiv:1704.00708 (2017) [7] how does Netflix recommend movies? [8] Li, Dongsheng, et al. "Mixture-Rank Matrix Approximation for Collaborative Filtering." Advances in Neural Information Processing Systems. 2017. !42
REFERENCES [9] Candès, Emmanuel J., and Benjamin Recht. "Exact matrix completion via convex optimization." Foundations of Computational mathematics 9.6 (2009): 717. [10] Glendinning, R. H. "An evaluation of the ICM algorithm for image reconstruction." Journal of Statistical Computation and Simulation 31.3 (1989): 169-185. [11] http://sifter.org/~simon/journal/20061211.html [12] https://www.slideshare.net/ssuser62b35f/exact-matrix-completion-via-convex- optimization-slideppt [13] Lee, Joonseok, et al. "Local low-rank matrix approximation." International Conference on Machine Learning. 2013. [14] Sedhain, Suvash, et al. "Autorec:Autoencoders meet collaborative filtering." Proceedings of the 24th International Conference onWorldWideWeb.ACM, 2015. [15] Zheng,Yin, et al. "A neural autoregressive approach to collaborative filtering." arXiv preprint arXiv:1605.09477 (2016). [16] Fu, Mingsheng, et al. "Attention based collaborative filtering." Neurocomputing (2018). [17] Li, Dongsheng, et al. "Low-rank matrix approximation with stability." International Conference on Machine Learning. 2016. !43

More Related Content

PPTX
객체지향 프로그래밍 기본
by용호 최
 
PDF
ADRという考えを取り入れてみて
byinfinite_loop
 
PDF
PyTorch Introduction
byYash Kawdiya
 
PDF
Spark Summit EU talk by Ted Malaska
bySpark Summit
 
PPTX
APACHE SPARK.pptx
byDeepaThirumurugan
 
PPTX
Js: master prototypes
byBarak Drechsler
 
PDF
Spark Summit EU talk by Sebastian Schroeder and Ralf Sigmund
bySpark Summit
 
PDF
入門ClojureScript
bysohta
 
객체지향 프로그래밍 기본
by용호 최
 
ADRという考えを取り入れてみて
byinfinite_loop
 
PyTorch Introduction
byYash Kawdiya
 
Spark Summit EU talk by Ted Malaska
bySpark Summit
 
APACHE SPARK.pptx
byDeepaThirumurugan
 
Js: master prototypes
byBarak Drechsler
 
Spark Summit EU talk by Sebastian Schroeder and Ralf Sigmund
bySpark Summit
 
入門ClojureScript
bysohta
 

What's hot

PDF
우아한테크세미나-우아한멀티모듈
by용근 권
 
PDF
객체지향 개념 (쫌 아는체 하기)
bySeung-June Lee
 
PDF
Drools and jBPM 6 Overview
byMark Proctor
 
PDF
SSD: Single Shot MultiBox Detector (UPC Reading Group)
byUniversitat Politècnica de Catalunya
 
PPTX
Deep learning with keras
byMOHITKUMAR1379
 
PDF
Gartner 2021 Magic Quadrant for Cloud Database Management Systems.pdf
bymomirlan
 
PPTX
Deep dive de Amazon DynamoDB
byAmazon Web Services LATAM
 
PPTX
XgBoost.pptx
bysumankumar507
 
PPTX
TensorRT survey
byYi-Hsiu Hsu
 
PDF
Introducing Apache Airflow and how we are using it
byBruno Faria
 
PDF
Spark SQL
byJoud Khattab
 
PPTX
Deep deterministic policy gradient
bySlobodan Blazeski
 
PDF
Python testing using mock and pytest
bySuraj Deshmukh
 
PDF
A Visual Introduction to Event Sourcing and CQRS by Lorenzo Nicora
byOpenCredo
 
PPTX
Kafka timestamp offset
byDaeMyung Kang
 
PPTX
K-means Clustering
bySajib Sen
 
ODP
Drools 6 deep dive
byMario Fusco
 
PDF
Building Scalable and Personalized News Feed
byEyal Ronel
 
PDF
[232] TensorRT를 활용한 딥러닝 Inference 최적화
byNAVER D2
 
PPTX
URL Shortener-4.pptx
bymanishgupta683517
 
우아한테크세미나-우아한멀티모듈
by용근 권
 
객체지향 개념 (쫌 아는체 하기)
bySeung-June Lee
 
Drools and jBPM 6 Overview
byMark Proctor
 
SSD: Single Shot MultiBox Detector (UPC Reading Group)
byUniversitat Politècnica de Catalunya
 
Deep learning with keras
byMOHITKUMAR1379
 
Gartner 2021 Magic Quadrant for Cloud Database Management Systems.pdf
bymomirlan
 
Deep dive de Amazon DynamoDB
byAmazon Web Services LATAM
 
XgBoost.pptx
bysumankumar507
 
TensorRT survey
byYi-Hsiu Hsu
 
Introducing Apache Airflow and how we are using it
byBruno Faria
 
Spark SQL
byJoud Khattab
 
Deep deterministic policy gradient
bySlobodan Blazeski
 
Python testing using mock and pytest
bySuraj Deshmukh
 
A Visual Introduction to Event Sourcing and CQRS by Lorenzo Nicora
byOpenCredo
 
Kafka timestamp offset
byDaeMyung Kang
 
K-means Clustering
bySajib Sen
 
Drools 6 deep dive
byMario Fusco
 
Building Scalable and Personalized News Feed
byEyal Ronel
 
[232] TensorRT를 활용한 딥러닝 Inference 최적화
byNAVER D2
 
URL Shortener-4.pptx
bymanishgupta683517
 

Similar to Why biased matrix factorization works well?

PDF
Netflix Recommendations - Beyond the 5 Stars
byXavier Amatriain
 
PDF
Paris ML meetup
byYves Raimond
 
PDF
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
byAnoop Deoras
 
PDF
Netflix recommendation systems
byMina Tafreshi
 
PDF
Algorithmic Music Recommendations at Spotify
byChris Johnson
 
PDF
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
byXavier Amatriain
 
PDF
Matrix Factorization Technique for Recommender Systems
byAladejubelo Oluwashina
 
PDF
Building Large-scale Real-world Recommender Systems - Recsys2012 tutorial
byXavier Amatriain
 
PDF
MLConf - Emmys, Oscars & Machine Learning Algorithms at Netflix
byXavier Amatriain
 
PDF
Xavier amatriain, dir algorithms netflix m lconf 2013
byMLconf
 
PDF
Applications of Data Science in Netflix
byAlkaJoy2
 
PPTX
News recommender system from math model to production solution
byNikolay Karpov
 
PDF
Practical Data Science Workshop - Recommendation Systems - Collaborative Filt...
byChris Fregly
 
PPTX
acmsigtalkshare-121023190142-phpapp01.pptx
bydongchangim30
 
PDF
Cikm 2013 - Beyond Data From User Information to Business Value
byXavier Amatriain
 
DOCX
Mining Large Streams of User Data for PersonalizedRecommenda.docx
byARIV4
 
PDF
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
byXavier Amatriain
 
PDF
Parismlmeetupfinalslides 151209190037-lva1-app6892
bymercedes calderon
 
PDF
Machine Learning
byArthur Breitman
 
PDF
Fast matrix primitives for ranking, link-prediction and more
byDavid Gleich
 
Netflix Recommendations - Beyond the 5 Stars
byXavier Amatriain
 
Paris ML meetup
byYves Raimond
 
Tutorial on Deep Learning in Recommender System, Lars summer school 2019
byAnoop Deoras
 
Netflix recommendation systems
byMina Tafreshi
 
Algorithmic Music Recommendations at Spotify
byChris Johnson
 
Qcon SF 2013 - Machine Learning & Recommender Systems @ Netflix Scale
byXavier Amatriain
 
Matrix Factorization Technique for Recommender Systems
byAladejubelo Oluwashina
 
Building Large-scale Real-world Recommender Systems - Recsys2012 tutorial
byXavier Amatriain
 
MLConf - Emmys, Oscars & Machine Learning Algorithms at Netflix
byXavier Amatriain
 
Xavier amatriain, dir algorithms netflix m lconf 2013
byMLconf
 
Applications of Data Science in Netflix
byAlkaJoy2
 
News recommender system from math model to production solution
byNikolay Karpov
 
Practical Data Science Workshop - Recommendation Systems - Collaborative Filt...
byChris Fregly
 
acmsigtalkshare-121023190142-phpapp01.pptx
bydongchangim30
 
Cikm 2013 - Beyond Data From User Information to Business Value
byXavier Amatriain
 
Mining Large Streams of User Data for PersonalizedRecommenda.docx
byARIV4
 
BIG2016- Lessons Learned from building real-life user-focused Big Data systems
byXavier Amatriain
 
Parismlmeetupfinalslides 151209190037-lva1-app6892
bymercedes calderon
 
Machine Learning
byArthur Breitman
 
Fast matrix primitives for ranking, link-prediction and more
byDavid Gleich
 

More from Joonyoung Yi

PDF
Introduction to XGBoost
byJoonyoung Yi
 
PDF
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
byJoonyoung Yi
 
PDF
Introduction to Low-rank Matrix Completion
byJoonyoung Yi
 
PDF
A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE) Slide
byJoonyoung Yi
 
PDF
Exact Matrix Completion via Convex Optimization Slide (PPT)
byJoonyoung Yi
 
PDF
Low-rank Matrix Approximation with Stability
byJoonyoung Yi
 
PDF
Sparsity Normalization: Stabilizing the Expected Outputs of Deep Networks
byJoonyoung Yi
 
PDF
Dynamically Expandable Network (DEN)
byJoonyoung Yi
 
PDF
Mixture-Rank Matrix Approximation for Collaborative Filtering
byJoonyoung Yi
 
Introduction to XGBoost
byJoonyoung Yi
 
Introduction to MAML (Model Agnostic Meta Learning) with Discussions
byJoonyoung Yi
 
Introduction to Low-rank Matrix Completion
byJoonyoung Yi
 
A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE) Slide
byJoonyoung Yi
 
Exact Matrix Completion via Convex Optimization Slide (PPT)
byJoonyoung Yi
 
Low-rank Matrix Approximation with Stability
byJoonyoung Yi
 
Sparsity Normalization: Stabilizing the Expected Outputs of Deep Networks
byJoonyoung Yi
 
Dynamically Expandable Network (DEN)
byJoonyoung Yi
 
Mixture-Rank Matrix Approximation for Collaborative Filtering
byJoonyoung Yi
 

Recently uploaded

PDF
apidays Paris 2025 | Optimize your architecture for the next wave of AI-infus...
byapidays
 
PPTX
2ai_mockinterview.pptx hjijjjjjjjjjnhhbh
bypankajponia112
 
PPTX
RAG_classnoteswith having various ways of publishing the model.pptx
byanjaneyav00
 
PPTX
Pakistan Tableau User Group Event: A guide to Salesforce Certified Data Analyst
byWaqarAhmedShaikh8
 
PPTX
Big Data Analytics Introduction & Applications.pptx
byprabha345213
 
PPTX
SlideEgg_66421-Data Lifecycle Management-4-3.pptx
byakashsarkar0424
 
PDF
Azure Data Factory Overview for SSIS Developers.pdf
bySrinivas V
 
DOCX
AI_ML_Project_Ideas_Chennai_Presentation.docx
bytoptechdevelopers001
 
PDF
7th International Conference on Artificial Intelligence and Big Data (AIBD 2026)
bymlaij
 
PPTX
NLP Generative artificial intelligence trends
byBasantAgarwal17
 
PDF
(25633) PEDAMARAN_TIMUR-PANCAWARNA_TPS 1.pdf
bysoemaryheart
 
PPTX
LinkedIn Lead Generation expert for B2B Lead Generation
byMdBelalSharif1
 
PPTX
Chapter_3_v9.0.pptxChapter_3_v9.0.pptxChapter_3_v9.0.pptx
bylolagom166
 
PPTX
ict_presentation_final_final_final[1].pptx
bysaadabdulghafoor24
 
PPTX
Ch6 Automata & Complexity Theory 2016.pptx
bytechnologyn387
 
PPTX
Geographical Information System Also Known As GIS
bynionyeet
 
PPTX
1-FUEL IN A DIESEL ENGINE UNDERSTANDING AND KNOWLEDGE CONCEPTS
byMarkFenwick10
 
PDF
CSE550_ Lecture 02robosssssssssssstik.pdf
byfarfauziar
 
PDF
CLINICO-SOCIAL CASE DISCUSSION-antenatal
bymlaksh790
 
PPTX
Investment and Portfilio. Group no 1.pptx
byAhtshamUlHaq10
 
apidays Paris 2025 | Optimize your architecture for the next wave of AI-infus...
byapidays
 
2ai_mockinterview.pptx hjijjjjjjjjjnhhbh
bypankajponia112
 
RAG_classnoteswith having various ways of publishing the model.pptx
byanjaneyav00
 
Pakistan Tableau User Group Event: A guide to Salesforce Certified Data Analyst
byWaqarAhmedShaikh8
 
Big Data Analytics Introduction & Applications.pptx
byprabha345213
 
SlideEgg_66421-Data Lifecycle Management-4-3.pptx
byakashsarkar0424
 
Azure Data Factory Overview for SSIS Developers.pdf
bySrinivas V
 
AI_ML_Project_Ideas_Chennai_Presentation.docx
bytoptechdevelopers001
 
7th International Conference on Artificial Intelligence and Big Data (AIBD 2026)
bymlaij
 
NLP Generative artificial intelligence trends
byBasantAgarwal17
 
(25633) PEDAMARAN_TIMUR-PANCAWARNA_TPS 1.pdf
bysoemaryheart
 
LinkedIn Lead Generation expert for B2B Lead Generation
byMdBelalSharif1
 
Chapter_3_v9.0.pptxChapter_3_v9.0.pptxChapter_3_v9.0.pptx
bylolagom166
 
ict_presentation_final_final_final[1].pptx
bysaadabdulghafoor24
 
Ch6 Automata & Complexity Theory 2016.pptx
bytechnologyn387
 
Geographical Information System Also Known As GIS
bynionyeet
 
1-FUEL IN A DIESEL ENGINE UNDERSTANDING AND KNOWLEDGE CONCEPTS
byMarkFenwick10
 
CSE550_ Lecture 02robosssssssssssstik.pdf
byfarfauziar
 
CLINICO-SOCIAL CASE DISCUSSION-antenatal
bymlaksh790
 
Investment and Portfilio. Group no 1.pptx
byAhtshamUlHaq10
 

Why biased matrix factorization works well?

  • 1.
    Why Biased Matrix FactorizationWorks Well? 2018. 8. 29. JoonyoungYi joonyoung.yi@kaist.ac.kr
 Machine Learning & Intelligence Laboratory School of Computing
 Korea Advanced Institute of Science andTechnology
  • 2.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 3.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 4.
    NETFLIX PRIZE • InOctober 2016, Netflix open a $1M Prize called Netflix Prize. • To improve recommendation accuracy by 10%. • They splitted the data for prize. • Training set: 100M • Test set: 1.4M • Training set contains 1.2 % of entries. • 480,000 users and 17,700 movies. • From a matrix viewpoint, training set contains only 1.2% entries of the matrix. !4 1 2 3 4 5 6 7 8 … 1 4 2 4 2 3 3 3 3 3 1 4 2 4 1 5 … 2 3 1 Users Movies 480,000 17,700
  • 5.
    DATA DESCRIPTION OFNETFLIX PRIZE • Each set(training set, test set) is a set of (i, j, Mij)s. • i: user index • j: item(or movie) index • Mij: true rating of user u to item i. • : the set of (i, j) corresponding to the training set(=The set of known entries). • : the set of (i, j) corresponding to the test set(=The set of entries to predict). • m is the number of users. • n is the number of items. !5 ⌦<latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit> ⌦0 <latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
  • 6.
    DATA DESCRIPTION OFNETFLIX PRIZE • RMSE(Root Mean Square Error) was used for measuring accuracy. • Rij: predicted rating of user i to item j. • is the cardinality of the . • Projection Operator • M, R are rating matrices. • !6 ⌦0 <latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit> |⌦0 |<latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit> train RMSE = s X ⌦ (Mij Rij)2 |⌦| = kP⌦(M R)kF p ⌦ test RMSE = s X ⌦0 (Mij Rij)2 |⌦0| = kP⌦0 (M R)kF p ⌦0 <latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit> P⌦(M) = ⇢ Mij if (i, j) 2 ⌦, 0 otherwise.<latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit> M, R 2 Rm⇥n <latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit>
  • 7.
    RMSE on TestSet RMSE OF NETFLIX, PRIZE WINNER AND BIASED-MF • RMSE of Cinematch (Original Netflix Recommendation Engine): 0.9514 • RMSE of Winner: 0.8553 (10% enhancement) • They ensembled a lot of algorithms. • RMSE of Biased-MF ONLY: 0.8799 (7.5% enhancement) • Some materials said 6% enhancement only by Biased-MF. • RMSE of Biased-MF(with tuned hyper-parameters): 0.844 (11.3% enhancement) • From the AutoRec Paper in WWW’15. !7
  • 8.
    BIASED-MF IS FASTAND WORKS WELL • Many Algorithms are presented after Biased-MF. • Biased-MF is the fastest algorithm among them. • The convex relaxation(nuclear norm minimization) algorithm is slower than Biased-MF. • Biased-MF works quite well even these days. • And, the some of algorithms with good performance are based on Biased-MF. • SMA and MRMA are ensemble algorithms based on Biased-MF. • LLoRMA and ABCF are modification of Biased-MF. !8 Movielens 1M Movielens 10M Netflix(100M) Biased MF(IEEE’09) 0.845 0.803 0.844 LLoRMA(ICML’13) 0.8333 0.7815 0.8337 AutoRec(WWW’15) 0.831 0.782 0.823 CF-NADE(ICML ’16) 0.829 0.771 0.803 SMA(ICML’16) - 0.7682 0.8036 ABCF(Neurocomputing’18) 0.836 0.766 0.795 MRMA(NIPS’17) - 0.7634 0.7973
  • 9.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 10.
    FUNK SVD(SINGULAR VECTORDECOMPOSITION) • Before introducing Biased-MF, Let’s see Funk SVD(suggested by Funk). • Goal of Netflix Prize: Minimize RMSE • There is no way to solve this optimization form without any assumption. • In rating recommendation, it is natural to assume the matrix is low-rank. • New Optimization Form with Low-rank Assumption: • p and q are low-rank matrices: • To avoid overfitting: !10 ˆrui<latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit> arg min R v u u t X (i,j)2⌦ (Mij Rij)2 |⌦| = arg min R X (i,j)2⌦ (Mij Rij)2 <latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 <latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit> max(rank( ˆU), rank( ˆV )) ⌧ dim(M)<latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit>
  • 11.
    SOLUTION OF THEFUNK SVD • Optimization form of the Funk SVD: • If we know all entries (= contains all (u, i) pairs), we can solve by SVD (Singular vector decomposition). • However, we can’t know the all entries. • If we don’t know all entries, this is NP-hard problem. • Then, how to solve this optimization problem? • One of approaches is convex relaxation. • But, the Funk-SVD use another algorithm. !11 ⌦<latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit> arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
  • 12.
    SOLVE BY ALTERNATINGMINIMIZATION !12 M ≒ xm m n n k k These can be solved by SVD! (pseudo inverse) ˆU<latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit> ˆV † <latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit> 1: Input: observed set ⌦, values P⌦(M) 2: Initialize ˆV 0 randomly. 3: for t = 1, · · · , T: 4: ˆUt arg min U2Rm⇥k kP⌦(M U( ˆV t 1 )† )k2 F + U kUk2 F 5: ˆV t arg min V 2Rk⇥n kP⌦(M ˆUt V † )k2 F + V kV k2 F 6: Return R = ˆUt ( ˆV t )† <latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit>
  • 13.
    WHAT IS BIASEDMATRIX FACTORIZATION? • The Biased Matrix Factorization is based on Funk SVD. • Biased MF = Funk SVD + Bias Terms • Some users tend to give higher ratings, and some users tend to give lower ratings. • The tendencies of movies are similar to that of users. • Some movies tend to get higher ratings, and some movies tend to get lower ratings. • Biased Matrix Factorization wants to handle this phenomenon. • To introduce bias terms in the optimization form. !13
  • 14.
    OPTIMIZATION FORM OFBIASED-MF • Therefore, Biased-MF introduce biased terms related to user and item respectively. • is the average rating of training set(constant). • is the bias vector related to user. • is the bias vector related to item. • This optimization form can be solved by alternating minimization. • Similar to the solution of Funk SVD. • The Biased Matrix Factorization is also called SVD++. !14 µ<latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit> buser <latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit> bitem <latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit> min ˆU, ˆV ,buser,bitem X (i,j)2⌦ (Mij µ buser i bitem j ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F + buser kbuser k2 2 + bitem kbitem k2 2<latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit>
  • 15.
    THE ROLE OFBIAS TERMS IN BIASED-MF • The bias terms serve the role of normalization. • The bias terms serve to make the rating matrix r be zero mean. • Meanwhile, the Biased-MF without bias terms(=Funk SVD) also works quite well. • The RMSE performance on Netflix Dataset is similar to original recommendation engine of Netflix. • Therefore, knowing why Biased-MF works well is equivalent to
 knowing why Funk SVD works well. • Now, we will investigate why Funk SVD works well on low-rank matrices. !15
  • 16.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 17.
    OPTIMIZATION FORM OFFUNK SVD • Recall: Optimization form of Funk SVD • Probabilistic Matrix Factorization (PMF, NIPS’08) interprets Funk SVD 
 in the view point of posterior. • Funk SVD can be interpreted as an algorithm that finds the most probable low-rank matrices p and q when a star is observed. !17 arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
  • 18.
    PROBABILISTIC MATRIX FACTORIZATION •Optimization form of PMF: • To find and via MAP(Maximum A Posteriori). • A low-rank assumption ( ) and Gaussian noise assumptions. !18 Mij i=1,…,m j=1,…,n σ σU σV arg max ˆU, ˆV Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] <latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit> ˆUi<latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit> ˆVj<latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit> Mij = ˆUi ˆV † j<latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit> Pr[M| ˆU, ˆV , 2 ] = mY i=1 nY j=1 [N[Mij| ˆUi ˆV † j , 2 ]]I⌦ ij , Pr[ ˆU| 2 U ] = mY i=1 N[ ˆUi|0, 2 U I], Pr[ ˆV | 2 V ] = nY j=1 N[ ˆVj|0, 2 V I], I⌦ ij = ⇢ 1 (i, j) 2 ⌦ 0 otherwise.<latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit> ̂U ̂V
  • 19.
    THE LOG POSTERIORIOF PMF • The log posteriori of PMF is as follows: • Hence, the following equation holds: !19 arg max ˆU, ˆV Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] = arg min ˆU, ˆV X (i,j)2⌦ (Mij ˆUi ˆV † j )2 + U k ˆUk2 F + V k ˆV k2 F <latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit> ln Pr[ ˆU, ˆV |P⌦(M), ⌦, 2 , 2 U , 2 V ] = 1 2 2 mX i=1 nX j=1 I⌦ ij(Mij ˆUi ˆV † j )2 1 2 2 U mX i=1 ˆU† i ˆUi 1 2 2 V nX j=1 ˆV † j ˆVj 1 2 (( mX i=1 nX j=1 I⌦ ij) ln 2 + mk ln 2 U + nk ln 2 V ) + C <latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit>
  • 20.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 21.
    WHY WE USEALTERNATING MINIMIZATION? • We can just use Gradient Descent with random initialization without alternating minimization. • Why we should use alternating minimization? • First of all, Gradient Descent can oscillate without alternating minimization. • The MRMA paper(NIPS’17) said that it is possible to overfit without alternating minimization. • Alternating Minimization is not the only way to avoid overfitting. • However, Besag(1986) said that it is a good way to avoid overfitting. • Glendinning(1989) said that alternating minimization is robust to initial point empirically. • However, the initial point is nonetheless important. • Because it can fall into the local minima. !21
  • 22.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 23.
    LOCAL MINIMA PROBLEMOF AM • In otherwise, Jain et al(2013) showed that AM has no local minima problem. • Noiseless case ONLY. Exact Completion Setting. • With the slightly modified algorithm. • With some assumptions. • Let’s look at the modified algorithm first. !23
  • 24.
  • 25.
  • 26.
    THE MODIFIED ALGORITHM !26 AlternatingMinimization similar to Funk SVD
  • 27.
    THE MODIFIED ALGORITHM !27 Whenperforming initialization using SVD + Clipping, the local optimum found by the SVD method
 is close enough to the global optimum!
  • 28.
    MOTIVATION OF THEINCOHERENCE ASSUMPTION • Consider the rank-1 matrix M: • Let |Ω| be the number of observed entries of M. • Then, we can see only 0 with probability 1 - |Ω| / (mn). • If sample set doesn’t contain 1, we can not complete matrix exactly. • Therefore, it is impossible to recover all low-rank matrices. !28 M = e1e⇤ n = 2 6 6 6 4 0 0 · · · 0 1 0 0 · · · 0 0 ... ... ... ... ... 0 0 · · · 0 0 3 7 7 7 5 <latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit>
  • 29.
    THE INCOHERENCE ASSUMPTION •More generally, it is hard to recover if the singular vectors of the matrix M are similar to standard basis. • Because, information is highly concentrated on specific region. • Hence, the singular vectors need to be sufficiently spread. • This is why the paper introduce the incoherence assumption. • The EMCCO paper said that the random low-rank (orthogonal) matrices satisfy the incoherent assumption. !29
  • 30.
    MAIN RESULTS • Requiredentries: |Ω| = O((k4.5 log k) n log n) • Required steps: O(log (1/ε)) • The EMCCO paper said that the optimum number of required entries is O(n log n) !30
  • 31.
    COMPARISON TO NUCLEARNORM MINIMIZATION • Alternating Minimization • Required entries: |Ω| = O((k4.5 log k) n log n) • Required steps: O(log (1/ε)) • Nuclear Norm Minimization (Convex Relaxation) • Required entries: |Ω| = O((k) n log n) • Required steps: O(ε-1/2) • Nuclear Norm Minimization requires smaller the number of samples. • Alternating Minimization converges faster than Nuclear Norm Minimization. • Theoretical bound is not tight in Nuclear Norm Minimization. !31
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
    TABLE OF CONTENTS 1.Netflix Prize and Winner’s Algorithm 2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009) 3. Probabilistic Matrix Factorization (NIPS 2008) 4. Why We Use Alternating Minimization? 5. Overcoming Local Minima Problem of Alternating Minimization 6. Why Random Initialization Works?
  • 38.
    WHY RANDOM INITIALIZATIONWORKS? • In prior paper, initialization steps matters to guarantee global optimality. • In practice, it is hard to determine hyper-parameters for clipping. • However, random initialization works well practically. • In this section, I’ll show you to why random initialization works. • It can be view as removing hyper-parameters. • Similar to motivation of WGAN-GP. • WGAN is really hard to determine hyper-parameter that performance is highly sensitive to. !38
  • 39.
    PRELIMINARY: MATRIX SENSINGPROBLEM • Quite similar to the matrix completion problem. • A Low-rank version of linear regression (y = xβ problem). • RIP condition: • Actually, the Jain et al’s work (2013) proved in low-rank matrix sensing problem first. • And then, they extended their work to low-rank matrix completion problem. !39
  • 40.
    NO SPURIOUS LOCALMINIMA • Rong et al (NIPS 2016), Matrix Completion has No Spurious Local Minimum • Matrix Completion problem with Semi-definite matrix assumption. • Showed why random initialization works with proper regularizer. • Srinadh et al (NIPS 2016), Global Optimality of Local Search for Low Rank Matrix Recovery • Matrix Sensing Problem. • Showed all local minima are very close to a global optimum with noisy measurements. • With a curvature bound (RIP condition), 
 a polynomial time global convergence is guaranteed by SGD from random initialization. • Rong et al (ArXiv 2017), No Spurious Local Minima in Non-convex Low Rank Problems: 
 A Unified Geometric Analysis • Extended Srinadh et al’s works from matrix sensing to matrix completion and robust PCA. • Showed no high-order saddle point exists if being with proper regularizer. !40
  • 41.
  • 42.
    REFERENCES [1] Koren,Yehuda, RobertBell, and ChrisVolinsky. "Matrix factorization techniques for recommender systems." Computer 8 (2009): 30-37. [2] Mnih,Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances in neural information processing systems. 2008. [3] Jain, Prateek, Praneeth Netrapalli, and Sujay Sanghavi. "Low-rank matrix completion using alternating minimization." Proceedings of the forty-fifth annual ACM symposium on Theory of computing.ACM, 2013. [4] Ge, Rong, Jason D. Lee, and Tengyu Ma. "Matrix completion has no spurious local minimum." Advances in Neural Information Processing Systems. 2016. [5] Bhojanapalli, Srinadh, Behnam Neyshabur, and Nati Srebro. "Global optimality of local search for low rank matrix recovery." Advances in Neural Information Processing Systems. 2016. [6] Ge, Rong, Chi Jin, andYi Zheng. "No spurious local minima in nonconvex low rank problems:A unified geometric analysis." arXiv preprint arXiv:1704.00708 (2017) [7] how does Netflix recommend movies? [8] Li, Dongsheng, et al. "Mixture-Rank Matrix Approximation for Collaborative Filtering." Advances in Neural Information Processing Systems. 2017. !42
  • 43.
    REFERENCES [9] Candès, EmmanuelJ., and Benjamin Recht. "Exact matrix completion via convex optimization." Foundations of Computational mathematics 9.6 (2009): 717. [10] Glendinning, R. H. "An evaluation of the ICM algorithm for image reconstruction." Journal of Statistical Computation and Simulation 31.3 (1989): 169-185. [11] http://sifter.org/~simon/journal/20061211.html [12] https://www.slideshare.net/ssuser62b35f/exact-matrix-completion-via-convex- optimization-slideppt [13] Lee, Joonseok, et al. "Local low-rank matrix approximation." International Conference on Machine Learning. 2013. [14] Sedhain, Suvash, et al. "Autorec:Autoencoders meet collaborative filtering." Proceedings of the 24th International Conference onWorldWideWeb.ACM, 2015. [15] Zheng,Yin, et al. "A neural autoregressive approach to collaborative filtering." arXiv preprint arXiv:1605.09477 (2016). [16] Fu, Mingsheng, et al. "Attention based collaborative filtering." Neurocomputing (2018). [17] Li, Dongsheng, et al. "Low-rank matrix approximation with stability." International Conference on Machine Learning. 2016. !43