The document discusses several techniques for collaborative filtering and recommendation systems including matrix factorization, convolutional matrix factorization (ConvMF), factorization machines, Bayesian probabilistic matrix factorization (BPMF), and Bayesian personalized ranking (BPR). Matrix factorization decomposes user-item matrices into latent factor vectors to make predictions. ConvMF extends MF by applying a convolutional neural network to model document context. Factorization machines and BPR are techniques for implicit feedback modeling and ranking. BPMF applies Bayesian inference to MF with Markov chain Monte Carlo sampling.

1. 1

2. ■ Matrix Factorization
■ Context-Aware MF
■ Factorization Machines
■ Bayesian Probabilistic Matrix Factorization
■ Bayesian Personalized Ranking
2

3. Matrix Factorization
■ User-item
– user item
■ Matrix Factorization user item
3
×
User-item User Item

4. Matrix Factorization
■ user item
user item
■ SGD
4
×
Vector dimension k

5. Matrix Factorization
■ user item
5
User a
Item 2
Item 3
User
user
Item
item
User d

6. Matrix Factorization
6
User a
Item 2
Item 3
User
Item
user item
User d
user
item
■ user item

7. Matrix Factorization
7
User a
Item 2
Item 3
User
user
User d
User a
Item 2
Item 3
User d
Item
item

8. Matrix Factorization
■
8
• k
•
User :
Item :

9. Matrix Factorization
9
U
R
V
!
"
#$
%
#&
%
#%

10. Matrix Factorization
10
U
R
V
!
"
#$
%
#&
%
#%
User

11. Matrix Factorization
11
U
R
V
!
"
#$
%
#&
%
#%
Item

12. Matrix Factorization
12
U
R
V
! "
#$
%
#&
%
#%

13. Matrix Factorization
■
13
SVD

14. Matrix Factorization
■
14
! "
1 0
#$
%&

15. Matrix Factorization
■
15

16. Convolutional Matrix Factorization
for Document Context-Aware
Recommendation
Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, Hwanjo Yu
16
2016, RecSys

17. ConvMF
■ Item
■ Item
17
…

18. ConvMF
18
U
R
V
!
"
#$
%
#&
%
#%
'
X
W
#(
%

19. ConvMF
19
U
R
V
!
"
#$
%
#&
%
#%
'
X
W
#(
%
MF CNN

20. ConvMF
20
U
R
V
!
"
#$
%
#&
%
#%
'
X
W
#(
%
CNN

21. ConvMF
21
U
R
V
!
"
#$
%
#&
%
#%
'
X
W
#(
%
Item

22. ConvMF
22
CNN 4
1. Embedding layer
2. Convolution layer
3. Pooling layer
4. Output layer

23. ConvMF
1. Embedding layer
23
Document
!! − 1 ! + 1
n
n Glove
2

24. ConvMF
2. Convolutional layer
24
!
…
Document Matrix
Window size "#
1 1
Contextual feature
$%

25. ConvMF
3. Pooling layer
25
n Contextual feature
n Document
max-pooling

26. ConvMF
4. Output layer
26
User & Item !

27. ConvMF
■
27
SGD + Back-prop

28. ConvMF
■
28
CNN

29. ConvMF
■
29
u
u p PMF :
p CTR : PMF + LDA
p CDL : PMF + SDAE
p ConvMF :
p ConvMF+ : Glove

30. ConvMF
■
30
RMSE

31. ConvMF
■
■
31

32. Factorization Machines
Steffen Rendle
32
2010, IEEE International Conference on Data Mining

33. FMs
■ Matrix Factorization
■ MF User Item
–
33

34. FMs
■
34
1.
User Item Time Rating
2. one-hot encoding
User Item Time Rating

35. FMs
■
35

36. FMs
■
36

37. FMs
■
37
User
Item

38. FMs
■ Alice Titanic 2010 1
38

39. FMs
39
Alice Titanic
2010 1 /

40. FMs
40
Alice Titanic
Alice Titanic
Alice 2010 1
Titanic 2010 1

41. FMs
■
41
! …
" …
"
!

42. FMs
■
– 3
– Ranking Binary Classification
–
Field-Aware Factorization Machines
42
[1]
[1] Field-Aware Factorization Machines for CTR Prediction
Y. Juan, Y. Zhuang, W. -S. Chin, C. -J. Lin
2016, RecSys

43. FMs
■ Domain-specific
– Matrix Factorization / Tensor decomposition
– SVD++
– Pairwise Interaction Tensor Factorization
– Factorized Personalized Markov Chains
or
43

44. Bayesian Probabilistic Matrix
Factorization using
Markov Chain Monte Carlo
Ruslan Salakhutdinov, Andriy Mnih
44
2008, ACM

45. BPMF
■ Matrix Factorization User Item
!"
#
!$
#
■
■
■
–
–
–
45

46. BPMF
46
U
R
V
!
"
#$
Λ& Λ'
(& ('()
*), ,)
*), ,)
()

47. BPMF
47
U
R
V
!
"
#$
Λ& Λ'
(& ('
)*, ,*
)*, ,*
(*
(*
User

48. BPMF
48
U
R
V
!
"
#$
Λ& Λ'
(& ('
)*, ,*
)*, ,*
(*
(*
User

49. BPMF
■ MCMC Gibbs sampling …
■
■
49

50. BPMF
■
50
u
u
p Netflix baseline : Netflix
p SVD
p PMF
p Logistic PMF : 0~1
p Bayesian PMF :

51. BPMF
■
51

52. BPMF
■
52

53. BPMF
■ User Item
53

54. BPMF
■ User Item
54

55. Bayesian Personalized
Ranking from Implicit
Feedback
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme
55
2009, UAI

56. BPR
■ Matrix Factorization
■
■
56

57. BPR
■ BPR
57

58. BPR
■ BPR
58
Θ
" #
$
"
# $

59. BPR
■
59
BPR

60. BPR
■
60

61. BPR
■ MF
61

62. BPR
■ AUC
62
AUC 0.94
0.77
0.49
0.26
1
1
1
0
…
…
1.

63. BPR
■ AUC
63
3. AUC
AUC
2. 1
0

64. BPR
■ AUC
64
AUC
0.94
0.77
0.49
0.26
1
1
1
0
…
…
…
0
1
1→0
0→1

65. BPR
■ AUC
65

66. BPR
■ AUC
66
!
" #
BPR

67. BPR
■
67
u
u
p WR-MF : 1 [0, 1]
p SVD-MF
p Cosine-kNN
p most popular :
p np_max :

68. BPR
■
68
AUC

69. Reference
■
■ Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, Hwanjo Yu.
Convolutional Matrix Factorization for Document Context-Aware
Recommendation, 2016, RecSys
■ Steffen Rendle. Factorization Machines, 2010, IEEE International Conference
on Data Mining
■ Ruslan Salakhutdinov, Andriy Mnih. Bayesian Probabilistic Matrix Factorization
using Markov Chain Monte Carlo, 2008, ACM
■ Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme.
Bayesian Personalized Ranking from Implicit Feedback, 2009, UAI
69