Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RSDL2016

640 views

Published on

Slides for RecSys 2016 Deep Learning workshop

Published in: Science
  • Be the first to comment

  • Be the first to like this

RSDL2016

  1. 1. Exploring Deep Space: Learning Personalized Ranking in a Semantic Space Jeroen Vuurens - Martha Larson - Arjen de Vries 1 https://arxiv.org/pdf/1608.00276v2 Star Wars IV Terminator 2 The Matrix f ( x )
  2. 2. Semantic spaces 2 image: https://www.tensorflow.org/versions/r0.10/tutorials/word2vec/index.html Consistent encoding of relations [2,3] [2] T. Mikolov, and J. Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 2013. [3] T. Mikolov, W.-T. Yih, and G. Zweig. Linguistic Regularities in Continuous Space Word man - woman country - capital
  3. 3. Semantic spaces 3 Consistent encoding of relations [2,3] [2] T. Mikolov, and J. Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 2013. [3] T. Mikolov, W.-T. Yih, and G. Zweig. Linguistic Regularities in Continuous Space Word
  4. 4. Semantic spaces 4 Consistent encoding of relations Similar for items? e.g. genre, suspense, strong language, suitability for children
  5. 5. Semantic Spaces 5 SciFi Action Drama Romance Thriller Fantasy War Crime
  6. 6. Semantic Spaces 6 SuspenseRealism 90’s Spielberg Harrison Ford
  7. 7. Ranking items 7 Star Wars IV #5 Terminator 2 #4 The Matrix #5 Star Wars V, VI Men in BlackJurassic ParkBack to the Future Raiders of the Lost Ark f(x)
  8. 8. Ranking items 8 f(x) Hyperplane + -
  9. 9. Ranking items 9 Scary + -
  10. 10. Ranking items 10 Scary likes dislikes indifferent f(x) f(x)
  11. 11. Implementation: learning vectors 11 [4] Q. V. Le and T. Mikolov. Distributed representations of sentences and documents. In Proceedings of ICML, 2014. ParagraphVector-DBOW [4], movieId userId_rating Hierarchical Softmax
  12. 12. Implementation: learning vectors 12 [4] Q. V. Le and T. Mikolov. Distributed representations of sentences and documents. In Proceedings of ICML, 2014. ParagraphVector-DBOW[4] Hierarchical Softmax Star Wars (user 3, rating 4) => user3_high
  13. 13. Implementation: learning vectors 13 [4] Q. V. Le and T. Mikolov. Distributed representations of sentences and documents. In Proceedings of ICML, 2014. PV-DBOW [4], content-based movieId wordInImdbReview Hierarchical Softmax
  14. 14. Implementation: ranking items 14 lower rated movie vector higher rated movie vector hyperplane coefficients distance hyperplane
  15. 15. Implementation: ranking items 15 lower rated movie vector higher rated movie vector hyperplane coefficients update:
  16. 16. Implementation: ranking items 16 lower rated movie vector higher rated movie vector hyperplane coefficients
  17. 17. Evaluation Movielens 1M (4k users, 6k movies) • Simulated online evaluation • 96% train, 2% validate, 2% test • Recall@10 for ratings >= 4 • compare against BPRMF, WRMF, UserKNN 17
  18. 18. Evaluation System Recall@10 sig. over* Popularity 0.053 BPRMF1 0.079 4 UserKNN2 0.087 4 WRMF3 0.089 4 DS-CF-500 0.144 1,2,3,4,5 DS-CF-1k 0.151 1,2,3,4,5 DS-CB-10k4 DS-VSM5 18 DS-CF: • item vectors learned from user-ratings • marginally reduces dimensionality • sig. more effective than other models * all p < 0.001
  19. 19. Evaluation System Recall@10 sig. over* Popularity 0.053 BPRMF1 0.079 4 UserKNN2 0.087 4 WRMF3 0.089 4 DS-CF-500 0.144 1,2,3,4,5 DS-CF-1k 0.151 1,2,3,4,5 DS-CB-10k4 0.075 DS-VSM5 19 DS-CB: • item vectors learned from IMDB user reviews • requires high dimensionality • potentially useful for novel items? * all p < 0.001
  20. 20. DS-VSM: • user-ratings used as item vector • ranking the items according to a hyperplane that optimally ranks user’s past ratings Evaluation System Recall@10 sig. over* Popularity 0.053 BPRMF1 0.079 4 UserKNN2 0.087 4 WRMF3 0.089 4 DS-CF-500 0.144 1,2,3,4,5 DS-CF-1k 0.151 1,2,3,4,5 DS-CB-10k4 0.075 DS-VSM5 0.119 1,2,3,4 20 * all p < 0.001
  21. 21. Analysis of parameters 21 Number of most recently rated items Dimensionality of the semantic space
  22. 22. Conclusion • Semantic item vectors encode substitutability • Rank items according to hyperplane, tuned to a user’s most recent N ratings. 22
  23. 23. Conclusion • Semantic item vectors encode substitutability • Rank items according to hyperplane, tuned to a user’s most recent N ratings. • Semantic space generalizes over the similarities between items 23
  24. 24. Conclusion • Semantic item vectors encode substitutability • Rank items according to hyperplane, tuned to a user’s most recent N ratings. • Semantic space generalizes preferences • Proposed pairwise L2R architecture allows to use high-dimensional latent vectors. 24
  25. 25. Questions? [1] W. Lowe. Towards a theory of semantic space. In Proceedings of CogSci, 2001. [2] T. Mikolov, and J. Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 2013. [3] T. Mikolov, W.-T. Yih, and G. Zweig. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of HLT-NAACL, 2013.paper: https://arxiv.org/abs/1608.00276
  26. 26. Compositionality of semantic spaces 26

×