The document discusses the Netflix Prize competition to improve movie recommendation accuracy, where the winning algorithm was based on biased matrix factorization. It provides background on the Netflix data and evaluation metrics used. Matrix factorization techniques like Funk SVD, probabilistic matrix factorization, and alternating minimization are introduced as important algorithms for collaborative filtering and recommendation systems.
Creating ML models is just the starting of a long journey. In this presentation which was given as a talk on e2e AI talks, I talk about the various challenges in the machine learning life cycle
Many state of the art machine learning applications today are based on artifical neural networks. In this talk we explore several commonly used neural network architectures. We identify the ideas behind their design, describe their topologies, outline their properties and discuss their use.
You might be enjoy this talk if you are interested in:
* Discovering some of the popular neural network types
* Learning about their design and how they work
* Understanding what are they are good for
Energy-based Model for Out-of-Distribution Detection in Deep Medical Image Se...Seunghyun Hwang
Presented work is accepted in Korean domestic conference, Korean Society of Artificial Intelligence in Medicine (KOSAIM) 2020, as a poster session.
- by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)
Creating ML models is just the starting of a long journey. In this presentation which was given as a talk on e2e AI talks, I talk about the various challenges in the machine learning life cycle
Many state of the art machine learning applications today are based on artifical neural networks. In this talk we explore several commonly used neural network architectures. We identify the ideas behind their design, describe their topologies, outline their properties and discuss their use.
You might be enjoy this talk if you are interested in:
* Discovering some of the popular neural network types
* Learning about their design and how they work
* Understanding what are they are good for
Energy-based Model for Out-of-Distribution Detection in Deep Medical Image Se...Seunghyun Hwang
Presented work is accepted in Korean domestic conference, Korean Society of Artificial Intelligence in Medicine (KOSAIM) 2020, as a poster session.
- by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)
In this Lunch & Learn session, Chirag Jain gives us a friendly & gentle introduction to Machine Learning & walks through High-Level Learning frameworks using Linear Classifiers.
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Preferred Networks
This presentation explains basic ideas of graph neural networks (GNNs) and their common applications. Primary target audiences are students, engineers and researchers who are new to GNNs but interested in using GNNs for their projects. This is a modified version of the course material for a special lecture on Data Science at Nara Institute of Science and Technology (NAIST), given by Preferred Networks researcher Katsuhiko Ishiguro, PhD.
Talk with Yves Raimond at the GPU Tech Conference on Marth 28, 2018 in San Jose, CA.
Abstract:
In this talk, we will survey how Deep Learning methods can be applied to personalization and recommendations. We will cover why standard Deep Learning approaches don't perform better than typical collaborative filtering techniques. Then we will survey we will go over recently published research at the intersection of Deep Learning and recommender systems, looking at how they integrate new types of data, explore new models, or change the recommendation problem statement. We will also highlight some of the ways that neural networks are used at Netflix and how we can use GPUs to train recommender systems. Finally, we will highlight promising new directions in this space.
This presentation on Recurrent Neural Network will help you understand what is a neural network, what are the popular neural networks, why we need recurrent neural network, what is a recurrent neural network, how does a RNN work, what is vanishing and exploding gradient problem, what is LSTM and you will also see a use case implementation of LSTM (Long short term memory). Neural networks used in Deep Learning consists of different layers connected to each other and work on the structure and functions of the human brain. It learns from huge volumes of data and used complex algorithms to train a neural net. The recurrent neural network works on the principle of saving the output of a layer and feeding this back to the input in order to predict the output of the layer. Now lets deep dive into this presentation and understand what is RNN and how does it actually work.
Below topics are explained in this recurrent neural networks tutorial:
1. What is a neural network?
2. Popular neural networks?
3. Why recurrent neural network?
4. What is a recurrent neural network?
5. How does an RNN work?
6. Vanishing and exploding gradient problem
7. Long short term memory (LSTM)
8. Use case implementation of LSTM
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you'll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
And according to payscale.com, the median salary for engineers with deep learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
Learn more at: https://www.simplilearn.com/
This Edureka Recurrent Neural Networks tutorial will help you in understanding why we need Recurrent Neural Networks (RNN) and what exactly it is. It also explains few issues with training a Recurrent Neural Network and how to overcome those challenges using LSTMs. The last section includes a use-case of LSTM to predict the next word using a sample short story
Below are the topics covered in this tutorial:
1. Why Not Feedforward Networks?
2. What Are Recurrent Neural Networks?
3. Training A Recurrent Neural Network
4. Issues With Recurrent Neural Networks - Vanishing And Exploding Gradient
5. Long Short-Term Memory Networks (LSTMs)
6. LSTM Use-Case
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckSlideTeam
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck is loaded with easy-to-follow content, and intuitive design. Introduce the types and levels of artificial intelligence using the highly-effective visuals featured in this PPT slide deck. Showcase the AI-subfield of machine learning, as well as deep learning through our comprehensive PowerPoint theme. Represent the differences, and interrelationship between AI, ML, and DL. Elaborate on the scope and use case of machine intelligence in healthcare, HR, banking, supply chain, or any other industry. Take advantage of the infographic-style layout to describe why AI is flourishing in today’s day and age. Elucidate AI trends such as robotic process automation, advanced cybersecurity, AI-powered chatbots, and more. Cover all the essentials of machine learning and deep learning with the help of this PPT slideshow. Outline the application, algorithms, use cases, significance, and selection criteria for machine learning. Highlight the deep learning process, types, limitations, and significance. Describe reinforcement training, neural network classifications, and a lot more. Hit download and begin personalization. Our AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck are topically designed to provide an attractive backdrop to any subject. Use them to look like a presentation pro. https://bit.ly/3ngJCKf
In this talk, Dmitry shares his approach to feature engineering which he used successfully in various Kaggle competitions. He covers common techniques used to convert your features into numeric representation used by ML algorithms.
Part 1 of the Deep Learning Fundamentals Series, this session discusses the use cases and scenarios surrounding Deep Learning and AI; reviews the fundamentals of artificial neural networks (ANNs) and perceptrons; discuss the basics around optimization beginning with the cost function, gradient descent, and backpropagation; and activation functions (including Sigmoid, TanH, and ReLU). The demos included in these slides are running on Keras with TensorFlow backend on Databricks.
What really are recommendations engines nowadays?
This presentation introduces the foundations of recommendation algorithms, and covers common approaches as well as some of the most advanced techniques. Although more focused on efficiency than theoretical properties, basics of matrix algebra and optimization-based machine learning are used through the presentation.
Table of Contents:
1. Collaborative Filtering
1.1 User-User
1.2 Item-Item
1.3 User-Item
* Matrix Factorization
* Stochastic Gradient Descent (SGD)
* Truncated Singular Value Decomposition (SVD)
* Alternating Least Square (ALS)
* Deep Learning
2. Content Extraction
* Item-Item Similarities
* Deep Content Extraction: NLP, CNN, LSTM
3. Hybrid Models
4. In Production
4.1 Problematics
4.2 Solutions
4.3 Tools
In this Lunch & Learn session, Chirag Jain gives us a friendly & gentle introduction to Machine Learning & walks through High-Level Learning frameworks using Linear Classifiers.
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Preferred Networks
This presentation explains basic ideas of graph neural networks (GNNs) and their common applications. Primary target audiences are students, engineers and researchers who are new to GNNs but interested in using GNNs for their projects. This is a modified version of the course material for a special lecture on Data Science at Nara Institute of Science and Technology (NAIST), given by Preferred Networks researcher Katsuhiko Ishiguro, PhD.
Talk with Yves Raimond at the GPU Tech Conference on Marth 28, 2018 in San Jose, CA.
Abstract:
In this talk, we will survey how Deep Learning methods can be applied to personalization and recommendations. We will cover why standard Deep Learning approaches don't perform better than typical collaborative filtering techniques. Then we will survey we will go over recently published research at the intersection of Deep Learning and recommender systems, looking at how they integrate new types of data, explore new models, or change the recommendation problem statement. We will also highlight some of the ways that neural networks are used at Netflix and how we can use GPUs to train recommender systems. Finally, we will highlight promising new directions in this space.
This presentation on Recurrent Neural Network will help you understand what is a neural network, what are the popular neural networks, why we need recurrent neural network, what is a recurrent neural network, how does a RNN work, what is vanishing and exploding gradient problem, what is LSTM and you will also see a use case implementation of LSTM (Long short term memory). Neural networks used in Deep Learning consists of different layers connected to each other and work on the structure and functions of the human brain. It learns from huge volumes of data and used complex algorithms to train a neural net. The recurrent neural network works on the principle of saving the output of a layer and feeding this back to the input in order to predict the output of the layer. Now lets deep dive into this presentation and understand what is RNN and how does it actually work.
Below topics are explained in this recurrent neural networks tutorial:
1. What is a neural network?
2. Popular neural networks?
3. Why recurrent neural network?
4. What is a recurrent neural network?
5. How does an RNN work?
6. Vanishing and exploding gradient problem
7. Long short term memory (LSTM)
8. Use case implementation of LSTM
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you'll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
And according to payscale.com, the median salary for engineers with deep learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
Learn more at: https://www.simplilearn.com/
This Edureka Recurrent Neural Networks tutorial will help you in understanding why we need Recurrent Neural Networks (RNN) and what exactly it is. It also explains few issues with training a Recurrent Neural Network and how to overcome those challenges using LSTMs. The last section includes a use-case of LSTM to predict the next word using a sample short story
Below are the topics covered in this tutorial:
1. Why Not Feedforward Networks?
2. What Are Recurrent Neural Networks?
3. Training A Recurrent Neural Network
4. Issues With Recurrent Neural Networks - Vanishing And Exploding Gradient
5. Long Short-Term Memory Networks (LSTMs)
6. LSTM Use-Case
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckSlideTeam
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck is loaded with easy-to-follow content, and intuitive design. Introduce the types and levels of artificial intelligence using the highly-effective visuals featured in this PPT slide deck. Showcase the AI-subfield of machine learning, as well as deep learning through our comprehensive PowerPoint theme. Represent the differences, and interrelationship between AI, ML, and DL. Elaborate on the scope and use case of machine intelligence in healthcare, HR, banking, supply chain, or any other industry. Take advantage of the infographic-style layout to describe why AI is flourishing in today’s day and age. Elucidate AI trends such as robotic process automation, advanced cybersecurity, AI-powered chatbots, and more. Cover all the essentials of machine learning and deep learning with the help of this PPT slideshow. Outline the application, algorithms, use cases, significance, and selection criteria for machine learning. Highlight the deep learning process, types, limitations, and significance. Describe reinforcement training, neural network classifications, and a lot more. Hit download and begin personalization. Our AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck are topically designed to provide an attractive backdrop to any subject. Use them to look like a presentation pro. https://bit.ly/3ngJCKf
In this talk, Dmitry shares his approach to feature engineering which he used successfully in various Kaggle competitions. He covers common techniques used to convert your features into numeric representation used by ML algorithms.
Part 1 of the Deep Learning Fundamentals Series, this session discusses the use cases and scenarios surrounding Deep Learning and AI; reviews the fundamentals of artificial neural networks (ANNs) and perceptrons; discuss the basics around optimization beginning with the cost function, gradient descent, and backpropagation; and activation functions (including Sigmoid, TanH, and ReLU). The demos included in these slides are running on Keras with TensorFlow backend on Databricks.
What really are recommendations engines nowadays?
This presentation introduces the foundations of recommendation algorithms, and covers common approaches as well as some of the most advanced techniques. Although more focused on efficiency than theoretical properties, basics of matrix algebra and optimization-based machine learning are used through the presentation.
Table of Contents:
1. Collaborative Filtering
1.1 User-User
1.2 Item-Item
1.3 User-Item
* Matrix Factorization
* Stochastic Gradient Descent (SGD)
* Truncated Singular Value Decomposition (SVD)
* Alternating Least Square (ALS)
* Deep Learning
2. Content Extraction
* Item-Item Similarities
* Deep Content Extraction: NLP, CNN, LSTM
3. Hybrid Models
4. In Production
4.1 Problematics
4.2 Solutions
4.3 Tools
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...Alessandro Suglia
Presentation for "Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neural Networks" at the 7th Italian Information Retrieval Workshop.
See paper: http://ceur-ws.org/Vol-1653/paper_11.pdf
Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neur...Claudio Greco
Slides for the presentation of the paper "Ask Me Any Rating: A Content-based Recommender System based on Recurrent Neural Networks" at the 7th Italian Information Retrieval Workshop.
Yuandong Tian at AI Frontiers : Planning in Reinforcement LearningAI Frontiers
Deep Reinforcement Learning (DRL) has made strong progress in many tasks, such as board games, robotics, navigation, neural architecture search, etc. I will present our recent open-sourced DRL frameworks to facilitate game research and development. Our framework is scalable so we can can reproduce AlphaGoZero and AlphaZero using 2000 GPUs, achieving super-human performance of Go AI that beats 4 top-30 professional players. We also show usability of our platform by training agents in real-time strategy games, and show interesting behaviors with a small amount of resource.
A business level introduction to Artificial Intelligence - Louis Dorard @ PAP...PAPIs.io
Artificial Intelligence and Machine Learning are becoming increasingly accessible. Starting from example use cases, I’ll aim at demystifying how they work and how they improve businesses in 3 areas: increasing the number of customers, serving them better, and serving them more efficiently. I’ll show how machines can use data to automatically learn business rules and make predictions, that can then be used to make better decisions. I’ll introduce the main concepts of ML, its possibilities, its limitations, and I’ll give tips on framing the right problems for your company to tackle.
Louis Dorard is the author of Bootstrapping Machine Learning, a co-founder of PAPIs, and an independent consultant. His goal is to help people use new machine learning technologies to make their apps and businesses smarter. He does this by writing, speaking and teaching.
The slide of the talk in http://www.meetup.com/R-Users-Sydney/events/223867196/
There is a web version here: http://wush978.github.io/FeatureHashing/index.html
This is the presentation for the paper "Fractional Step Discriminant Pruning: A Filter Pruning Framework for Deep Convolutional Neural Networks", delivered by N. Gkalelis and V. Mezaris at the 7th IEEE Int. Workshop on Mobile Multimedia Computing (MMC2020) that was held as part of the IEEE Int. Conf. on Multimedia and Expo (ICME), in July 2020.
Mixture-Rank Matrix Approximation for Collaborative FilteringJoonyoung Yi
The unofficial slide of Mixture-Rank Matrix Approximation for Collaborative Filtering (NIPS 2017)
--
Abstract of the paper: Low-rank matrix approximation (LRMA) methods have achieved excellent accuracy among today's collaborative filtering (CF) methods. In existing LRMA methods, the rank of user/item feature matrices is typically fixed, i.e., the same rank is adopted to describe all users/items. However, our studies show that submatrices with different ranks could coexist in the same user-item rating matrix, so that approximations with fixed ranks cannot perfectly describe the internal structures of the rating matrix, therefore leading to inferior recommendation accuracy. In this paper, a mixture-rank matrix approximation (MRMA) method is proposed, in which user-item ratings can be characterized by a mixture of LRMA models with different ranks. Meanwhile, a learning algorithm capitalizing on iterated condition modes is proposed to tackle the non-convex optimization problem pertaining to MRMA. Experimental studies on MovieLens and Netflix datasets demonstrate that MRMA can outperform six state-of-the-art LRMA-based CF methods in terms of recommendation accuracy.
Sparsity Normalization: Stabilizing the Expected Outputs of Deep NetworksJoonyoung Yi
The learning of deep models, in which a numerous of parameters are superimposed, is known to be a fairly sensitive process and should be carefully done through a combination of several techniques that can help to stabilize it. We introduce an additional challenge that has never been explicitly studied: the heterogeneity of sparsity at the instance level due to missing values or the innate nature of the input distribution. We confirm experimentally on the widely used benchmark datasets that this variable sparsity problem makes the output statistics of neurons unstable and makes the learning process more difficult by saturating non-linearities. We also provide the analysis of this phenomenon, and based on our analysis, we present a simple technique to prevent this issue, referred to as Sparsity Normalization (SN). Finally, we show that the performance can be significantly improved with SN on certain popular benchmark datasets, or that similar performance can be achieved with lower capacity. Especially focusing on the collaborative filtering problem where the variable sparsity issue has been completely ignored, we achieve new state-of-the-art results on Movielens 100k and 1M datasets, by simply applying Sparsity Normalization (SN).
https://arxiv.org/abs/1906.00150
Low-rank Matrix Approximation with StabilityJoonyoung Yi
Slide of Low-rank Matrix Approximation with Stability (SMA), ICML 2016
--
Abstract of Paper:
Low-rank matrix approximation has been widely
adopted in machine learning applications with sparse data, such as recommender systems. However, the sparsity of the data, incomplete and
noisy, introduces challenges to the algorithm stability – small changes in the training data may
significantly change the models. As a result, existing low-rank matrix approximation solutions yield low generalization performance, exhibiting high error variance on the training dataset,
and minimizing the training error may not guarantee error reduction on the testing dataset. In
this paper, we investigate the algorithm stability problem of low-rank matrix approximations.
We present a new algorithm design framework,
which (1) introduces new optimization objectives
to guide stable matrix approximation algorithm
design, and (2) solves the optimization problem
to obtain stable low-rank approximation solutions with good generalization performance. Experimental results on real-world datasets demonstrate that the proposed work can achieve better
prediction accuracy compared with both state-ofthe-art low-rank matrix approximation methods
and ensemble methods in recommendation task
A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE) Slide Joonyoung Yi
PPT for A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE).
I made ppt for explaining the paper.
Abstract of the paper:
This paper proposes CF-NADE, a neural autoregressive architecture for collaborative filtering (CF) tasks, which is inspired by the Restricted Boltzmann Machine (RBM) based CF model and the Neural Autoregressive Distribution Estimator (NADE). We first describe the basic CF-NADE model for CF tasks. Then we propose to improve the model by sharing parameters between different ratings. A factored version of CF-NADE is also proposed for better scalability. Furthermore, we take the ordinal nature of the preferences into consideration and propose an ordinal cost to optimize CF-NADE, which shows superior performance. Finally, CF-NADE can be extended to a deep model, with only moderately increased computational complexity. Experimental results show that CF-NADE with a single hidden layer beats all previous state-of-the-art methods on MovieLens 1M, MovieLens 10M, and Netflix datasets, and adding more hidden layers can further improve the performance.
Exact Matrix Completion via Convex Optimization Slide (PPT)Joonyoung Yi
Slide of the paper "Exact Matrix Completion via Convex Optimization" of Emmanuel J. Candès and Benjamin Recht. We presented this slide in KAIST CS592 Class, April 2018.
- Code: https://github.com/JoonyoungYi/MCCO-numpy
- Abstract of the paper: We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M. Can we complete the matrix and recover the entries that we have not seen? We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys
𝑚≥𝐶𝑛1.2𝑟log𝑛
for some positive numerical constant C, then with very high probability, most n×n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
1. Why Biased Matrix
Factorization Works Well?
2018. 8. 29.
JoonyoungYi
joonyoung.yi@kaist.ac.kr
Machine Learning & Intelligence Laboratory
School of Computing
Korea Advanced Institute of Science andTechnology
2. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
3. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
4. NETFLIX PRIZE
• In October 2016, Netflix open a $1M Prize called Netflix Prize.
• To improve recommendation accuracy by 10%.
• They splitted the data for prize.
• Training set: 100M
• Test set: 1.4M
• Training set contains 1.2 % of entries.
• 480,000 users and 17,700 movies.
• From a matrix viewpoint, training set contains only 1.2% entries of the matrix.
!4
1 2 3 4 5 6 7 8 …
1 4 2 4
2 3 3 3
3 3 1
4 2 4 1 5
…
2 3 1
Users
Movies
480,000
17,700
5. DATA DESCRIPTION OF NETFLIX PRIZE
• Each set(training set, test set) is a set of (i, j, Mij)s.
• i: user index
• j: item(or movie) index
• Mij: true rating of user u to item i.
• : the set of (i, j) corresponding to the training set(=The set of known entries).
• : the set of (i, j) corresponding to the test set(=The set of entries to predict).
• m is the number of users.
• n is the number of items.
!5
⌦<latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit>
⌦0
<latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
6. DATA DESCRIPTION OF NETFLIX PRIZE
• RMSE(Root Mean Square Error) was used for measuring accuracy.
• Rij: predicted rating of user i to item j.
• is the cardinality of the .
• Projection Operator
• M, R are rating matrices.
•
!6
⌦0
<latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
|⌦0
|<latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit>
train RMSE =
s
X
⌦
(Mij Rij)2
|⌦|
=
kP⌦(M R)kF
p
⌦
test RMSE =
s
X
⌦0
(Mij Rij)2
|⌦0|
=
kP⌦0 (M R)kF
p
⌦0
<latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit>
P⌦(M) =
⇢
Mij if (i, j) 2 ⌦,
0 otherwise.<latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit>
M, R 2 Rm⇥n
<latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit>
7. RMSE on Test Set
RMSE OF NETFLIX, PRIZE WINNER AND BIASED-MF
• RMSE of Cinematch (Original Netflix Recommendation Engine): 0.9514
• RMSE of Winner: 0.8553 (10% enhancement)
• They ensembled a lot of algorithms.
• RMSE of Biased-MF ONLY: 0.8799 (7.5% enhancement)
• Some materials said 6% enhancement only by Biased-MF.
• RMSE of Biased-MF(with tuned hyper-parameters): 0.844 (11.3% enhancement)
• From the AutoRec Paper in WWW’15.
!7
8. BIASED-MF IS FAST AND WORKS WELL
• Many Algorithms are presented after Biased-MF.
• Biased-MF is the fastest algorithm among them.
• The convex relaxation(nuclear norm minimization) algorithm is slower than Biased-MF.
• Biased-MF works quite well even these days.
• And, the some of algorithms with good performance are based on Biased-MF.
• SMA and MRMA are ensemble algorithms based on Biased-MF.
• LLoRMA and ABCF are modification of Biased-MF.
!8
Movielens 1M Movielens 10M Netflix(100M)
Biased MF(IEEE’09) 0.845 0.803 0.844
LLoRMA(ICML’13) 0.8333 0.7815 0.8337
AutoRec(WWW’15) 0.831 0.782 0.823
CF-NADE(ICML ’16) 0.829 0.771 0.803
SMA(ICML’16) - 0.7682 0.8036
ABCF(Neurocomputing’18) 0.836 0.766 0.795
MRMA(NIPS’17) - 0.7634 0.7973
9. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
10. FUNK SVD(SINGULAR VECTOR DECOMPOSITION)
• Before introducing Biased-MF, Let’s see Funk SVD(suggested by Funk).
• Goal of Netflix Prize: Minimize RMSE
• There is no way to solve this optimization form without any assumption.
• In rating recommendation, it is natural to assume the matrix is low-rank.
• New Optimization Form with Low-rank Assumption:
• p and q are low-rank matrices:
• To avoid overfitting:
!10
ˆrui<latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit>
arg min
R
v
u
u
t
X
(i,j)2⌦
(Mij Rij)2
|⌦|
= arg min
R
X
(i,j)2⌦
(Mij Rij)2
<latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit>
arg min
ˆU, ˆV
X
(i,j)2⌦
(Mij
ˆUi
ˆV †
j )2
<latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit>
arg min
ˆU, ˆV
X
(i,j)2⌦
(Mij
ˆUi
ˆV †
j )2
+ U k ˆUk2
F + V k ˆV k2
F
<latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
max(rank( ˆU), rank( ˆV )) ⌧ dim(M)<latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit>
11. SOLUTION OF THE FUNK SVD
• Optimization form of the Funk SVD:
• If we know all entries (= contains all (u, i) pairs), we can solve by SVD (Singular vector
decomposition).
• However, we can’t know the all entries.
• If we don’t know all entries, this is NP-hard problem.
• Then, how to solve this optimization problem?
• One of approaches is convex relaxation.
• But, the Funk-SVD use another algorithm.
!11
⌦<latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit>
arg min
ˆU, ˆV
X
(i,j)2⌦
(Mij
ˆUi
ˆV †
j )2
+ U k ˆUk2
F + V k ˆV k2
F
<latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
12. SOLVE BY ALTERNATING MINIMIZATION
!12
M ≒ xm m
n
n
k
k
These can be solved by SVD!
(pseudo inverse)
ˆU<latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit>
ˆV †
<latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit>
1: Input: observed set ⌦, values P⌦(M)
2: Initialize ˆV 0
randomly.
3: for t = 1, · · · , T:
4: ˆUt
arg min
U2Rm⇥k
kP⌦(M U( ˆV t 1
)†
)k2
F + U kUk2
F
5: ˆV t
arg min
V 2Rk⇥n
kP⌦(M ˆUt
V †
)k2
F + V kV k2
F
6: Return R = ˆUt
( ˆV t
)†
<latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit>
13. WHAT IS BIASED MATRIX FACTORIZATION?
• The Biased Matrix Factorization is based on Funk SVD.
• Biased MF = Funk SVD + Bias Terms
• Some users tend to give higher ratings, and some users tend to give lower ratings.
• The tendencies of movies are similar to that of users.
• Some movies tend to get higher ratings, and some movies tend to get lower ratings.
• Biased Matrix Factorization wants to handle this phenomenon.
• To introduce bias terms in the optimization form.
!13
14. OPTIMIZATION FORM OF BIASED-MF
• Therefore, Biased-MF introduce biased terms related to user and item respectively.
• is the average rating of training set(constant).
• is the bias vector related to user.
• is the bias vector related to item.
• This optimization form can be solved by alternating minimization.
• Similar to the solution of Funk SVD.
• The Biased Matrix Factorization is also called SVD++.
!14
µ<latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit>
buser
<latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit>
bitem
<latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit>
min
ˆU, ˆV ,buser,bitem
X
(i,j)2⌦
(Mij µ buser
i bitem
j
ˆUi
ˆV †
j )2
+ U k ˆUk2
F + V k ˆV k2
F + buser kbuser
k2
2 + bitem kbitem
k2
2<latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit>
15. THE ROLE OF BIAS TERMS IN BIASED-MF
• The bias terms serve the role of normalization.
• The bias terms serve to make the rating matrix r be zero mean.
• Meanwhile, the Biased-MF without bias terms(=Funk SVD) also works quite well.
• The RMSE performance on Netflix Dataset is similar to original recommendation engine of
Netflix.
• Therefore, knowing why Biased-MF works well is equivalent to
knowing why Funk SVD works well.
• Now, we will investigate why Funk SVD works well on low-rank matrices.
!15
16. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
17. OPTIMIZATION FORM OF FUNK SVD
• Recall: Optimization form of Funk SVD
• Probabilistic Matrix Factorization (PMF, NIPS’08) interprets Funk SVD
in the view point of posterior.
• Funk SVD can be interpreted as an algorithm that finds the most probable low-rank matrices
p and q when a star is observed.
!17
arg min
ˆU, ˆV
X
(i,j)2⌦
(Mij
ˆUi
ˆV †
j )2
+ U k ˆUk2
F + V k ˆV k2
F
<latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
18. PROBABILISTIC MATRIX FACTORIZATION
• Optimization form of PMF:
• To find and via MAP(Maximum A Posteriori).
• A low-rank assumption ( ) and Gaussian noise assumptions.
!18
Mij
i=1,…,m
j=1,…,n
σ
σU σV
arg max
ˆU, ˆV
Pr[ ˆU, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
<latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit>
ˆUi<latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit>
ˆVj<latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit>
Mij = ˆUi
ˆV †
j<latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit>
Pr[M| ˆU, ˆV , 2
] =
mY
i=1
nY
j=1
[N[Mij| ˆUi
ˆV †
j , 2
]]I⌦
ij ,
Pr[ ˆU| 2
U ] =
mY
i=1
N[ ˆUi|0, 2
U I],
Pr[ ˆV | 2
V ] =
nY
j=1
N[ ˆVj|0, 2
V I],
I⌦
ij =
⇢
1 (i, j) 2 ⌦
0 otherwise.<latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit>
̂U ̂V
19. THE LOG POSTERIORI OF PMF
• The log posteriori of PMF is as follows:
• Hence, the following equation holds:
!19
arg max
ˆU, ˆV
Pr[ ˆU, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
= arg min
ˆU, ˆV
X
(i,j)2⌦
(Mij
ˆUi
ˆV †
j )2
+ U k ˆUk2
F + V k ˆV k2
F
<latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit>
ln Pr[ ˆU, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
=
1
2 2
mX
i=1
nX
j=1
I⌦
ij(Mij
ˆUi
ˆV †
j )2 1
2 2
U
mX
i=1
ˆU†
i
ˆUi
1
2 2
V
nX
j=1
ˆV †
j
ˆVj
1
2
((
mX
i=1
nX
j=1
I⌦
ij) ln 2
+ mk ln 2
U + nk ln 2
V ) + C
<latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit>
20. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
21. WHY WE USE ALTERNATING MINIMIZATION?
• We can just use Gradient Descent with random initialization without alternating
minimization.
• Why we should use alternating minimization?
• First of all, Gradient Descent can oscillate without alternating minimization.
• The MRMA paper(NIPS’17) said that it is possible to overfit without alternating
minimization.
• Alternating Minimization is not the only way to avoid overfitting.
• However, Besag(1986) said that it is a good way to avoid overfitting.
• Glendinning(1989) said that alternating minimization is robust to initial point empirically.
• However, the initial point is nonetheless important.
• Because it can fall into the local minima.
!21
22. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
23. LOCAL MINIMA PROBLEM OF AM
• In otherwise, Jain et al(2013) showed that AM has no local minima problem.
• Noiseless case ONLY. Exact Completion Setting.
• With the slightly modified algorithm.
• With some assumptions.
• Let’s look at the modified algorithm first.
!23
27. THE MODIFIED ALGORITHM
!27
When performing initialization using SVD + Clipping,
the local optimum found by the SVD method
is close enough to the global optimum!
28. MOTIVATION OF THE INCOHERENCE ASSUMPTION
• Consider the rank-1 matrix M:
• Let |Ω| be the number of observed entries of M.
• Then, we can see only 0 with probability 1 - |Ω| / (mn).
• If sample set doesn’t contain 1, we can not complete matrix exactly.
• Therefore, it is impossible to recover all low-rank matrices.
!28
M = e1e⇤
n =
2
6
6
6
4
0 0 · · · 0 1
0 0 · · · 0 0
...
...
...
...
...
0 0 · · · 0 0
3
7
7
7
5
<latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit>
29. THE INCOHERENCE ASSUMPTION
• More generally, it is hard to recover if the singular vectors of the matrix M are similar to
standard basis.
• Because, information is highly concentrated on specific region.
• Hence, the singular vectors need to be sufficiently spread.
• This is why the paper introduce the incoherence assumption.
• The EMCCO paper said that the random low-rank (orthogonal) matrices satisfy the
incoherent assumption.
!29
30. MAIN RESULTS
• Required entries: |Ω| = O((k4.5 log k) n log n)
• Required steps: O(log (1/ε))
• The EMCCO paper said that the optimum number of required entries is O(n log n)
!30
31. COMPARISON TO NUCLEAR NORM MINIMIZATION
• Alternating Minimization
• Required entries: |Ω| = O((k4.5 log k) n log n)
• Required steps: O(log (1/ε))
• Nuclear Norm Minimization (Convex Relaxation)
• Required entries: |Ω| = O((k) n log n)
• Required steps: O(ε-1/2)
• Nuclear Norm Minimization requires smaller the number of samples.
• Alternating Minimization converges faster than Nuclear Norm Minimization.
• Theoretical bound is not tight in Nuclear Norm Minimization.
!31
37. TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?
38. WHY RANDOM INITIALIZATION WORKS?
• In prior paper, initialization steps matters to guarantee global optimality.
• In practice, it is hard to determine hyper-parameters for clipping.
• However, random initialization works well practically.
• In this section, I’ll show you to why random initialization works.
• It can be view as removing hyper-parameters.
• Similar to motivation of WGAN-GP.
• WGAN is really hard to determine hyper-parameter that performance is highly sensitive to.
!38
39. PRELIMINARY: MATRIX SENSING PROBLEM
• Quite similar to the matrix completion problem.
• A Low-rank version of linear regression (y = xβ problem).
• RIP condition:
• Actually, the Jain et al’s work (2013) proved in low-rank matrix sensing problem first.
• And then, they extended their work to low-rank matrix completion problem.
!39
40. NO SPURIOUS LOCAL MINIMA
• Rong et al (NIPS 2016), Matrix Completion has No Spurious Local Minimum
• Matrix Completion problem with Semi-definite matrix assumption.
• Showed why random initialization works with proper regularizer.
• Srinadh et al (NIPS 2016), Global Optimality of Local Search for Low Rank Matrix
Recovery
• Matrix Sensing Problem.
• Showed all local minima are very close to a global optimum with noisy measurements.
• With a curvature bound (RIP condition),
a polynomial time global convergence is guaranteed by SGD from random initialization.
• Rong et al (ArXiv 2017), No Spurious Local Minima in Non-convex Low Rank Problems:
A Unified Geometric Analysis
• Extended Srinadh et al’s works from matrix sensing to matrix completion and robust PCA.
• Showed no high-order saddle point exists if being with proper regularizer.
!40
42. REFERENCES
[1] Koren,Yehuda, Robert Bell, and ChrisVolinsky. "Matrix factorization techniques for
recommender systems." Computer 8 (2009): 30-37.
[2] Mnih,Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances
in neural information processing systems. 2008.
[3] Jain, Prateek, Praneeth Netrapalli, and Sujay Sanghavi. "Low-rank matrix completion
using alternating minimization." Proceedings of the forty-fifth annual ACM symposium on
Theory of computing.ACM, 2013.
[4] Ge, Rong, Jason D. Lee, and Tengyu Ma. "Matrix completion has no spurious local
minimum." Advances in Neural Information Processing Systems. 2016.
[5] Bhojanapalli, Srinadh, Behnam Neyshabur, and Nati Srebro. "Global optimality of local
search for low rank matrix recovery." Advances in Neural Information Processing Systems.
2016.
[6] Ge, Rong, Chi Jin, andYi Zheng. "No spurious local minima in nonconvex low rank
problems:A unified geometric analysis." arXiv preprint arXiv:1704.00708 (2017)
[7] how does Netflix recommend movies?
[8] Li, Dongsheng, et al. "Mixture-Rank Matrix Approximation for Collaborative Filtering."
Advances in Neural Information Processing Systems. 2017.
!42
43. REFERENCES
[9] Candès, Emmanuel J., and Benjamin Recht. "Exact matrix completion via convex
optimization." Foundations of Computational mathematics 9.6 (2009): 717.
[10] Glendinning, R. H. "An evaluation of the ICM algorithm for image reconstruction."
Journal of Statistical Computation and Simulation 31.3 (1989): 169-185.
[11] http://sifter.org/~simon/journal/20061211.html
[12] https://www.slideshare.net/ssuser62b35f/exact-matrix-completion-via-convex-
optimization-slideppt
[13] Lee, Joonseok, et al. "Local low-rank matrix approximation." International Conference on
Machine Learning. 2013.
[14] Sedhain, Suvash, et al. "Autorec:Autoencoders meet collaborative
filtering." Proceedings of the 24th International Conference onWorldWideWeb.ACM, 2015.
[15] Zheng,Yin, et al. "A neural autoregressive approach to collaborative filtering." arXiv
preprint arXiv:1605.09477 (2016).
[16] Fu, Mingsheng, et al. "Attention based collaborative filtering." Neurocomputing (2018).
[17] Li, Dongsheng, et al. "Low-rank matrix approximation with stability." International
Conference on Machine Learning. 2016.
!43