Modern Perspectives on Recommender Systems and their Applications in Mendeley

1. Modern Perspectives on Recommender Systems and their Applications in Mendeley Kris Jack and Maya Hristakeva 16/12/2014

2. Kris Jack, Chief Data Scientist http://www.mendeley.com/profiles/kris-jack/ Maya Hristakeva, Senior Data Scientist http://www.mendeley.com/profiles/maya-hristakeva/ Phil Gooch, Senior Data Scientist http://www.mendeley.com/profiles/phil-gooch/

3. Overview • The what and why of recommenders • Evolution of the recommender problem • Recommender algorithms • Evaluating a recommender • Recommender systems @ Mendeley

5. What is a recommender? A recommendation system (recommender) is a push system that presents users with the most relevant content for their context and needs • helps users to deal with information overload • recommenders are complementary to search search engine pull recommendation engine push Information Retrieval Information request infers context and needs Filtering

6. Recommenders @ Linkedin 50% of LinkedIn connections are from recommendations

7. Recommenders @ Linkedin

8. Recommenders @ Netflix Stop 1% of users from cancelling subscription = $500M/year Netflix invests $150M/year (300 people) in their content rec team

9. Recommenders @ ResearchGate

10. Why recommenders? • Search and recommendations are complementary, have arms and legs! • Higher usability, user satisfaction and engagement • Increase product stickiness • Monetise them ...and in the context of research... Help researchers keep up-to-date with latest research, connect with researchers in their field, contextualise their work within the global body of research (articles, researchers, conferences, research groups, etc.)

12. Evolution of recommender problem Problem: We have a massive collection of items (e.g. > 1 million). We want to recommend 5 items that the user will like.

13. Evolution of recommender problem First, seen as a ratings prediction problem. So, given some knowledge of the user, estimate how much they will appreciate each item on scale of 1-5. 4.9 choose top 5 items with highest predicted ratings 4.7 4.7 4.6 4.5

14. Evolution of recommender problem But do predicted ratings give the best order? Improve the recommender by reranking a selection of items with high predicted ratings. rerank items that are highly predicted 4.7 4.9 4.6 4.6 4.8

15. Evolution of recommender problem Let’s improve the recommendations by optimizing the page in which they appear. deliver them in style

16. Evolution of recommender problem Take the user’s context into account. new to this topic? no yes

17. Evolution of recommender problem Actively researching how to take other properties into account in context: trustworthiness; freshness; diversity; serendipity; novelty; recency. no at work? yes

18. Evolution of recommender problem Rating prediction Reranking Page optimisation Context-aware Future: trustworthiness; freshness; diversity; serendipity; novelty; recency. How to make recommendations? On to the algorithms... time

20. Recommender algorithms A recommender processes information and transforms it into actionable knowledge. Here we’ll focus on the algorithms that make this possible. information flow (components often built in parallel)

21. Recommender algorithms • Collaborative filtering (similarity and model-based) • Content-based filtering • Hybrid • Non-traditional

22. Collaborative filtering • User-based CF finds users who have similar appreciations for items as you and recommends new items based on what they like. • Item-based CF finds items that are similar to the ones you like. Similarity is based on item cooccurrences (e.g. the users who bought x also bought y). Formal representation Similarity-based CF • ti: rating of user xi for item yi. • Infer prediction function

23. Collaborative filtering • Matrix Factorisation (SVD++) • Clustering (K-means to LDA) • LSH (Locality sensitive hashing) • Restricted Boltzmann Machines Formal representation of MF • X: user-item ratings matrix • U: user-latent factors matrix • S: latent factor diagonal matrix • V: latent factor-item matrix Model-based CF

24. Collaborative filtering Pros • Minimal domain knowledge required • User and item features are not required • Produces good enough results in most cases Cons • Cold start problem • Requires high user:item ratio (1: 10) • Needs standardised products • Popularity bias (doesn’t play well with the long tail) • User-based CF • Item-based CF • Model-based CF

25. Content-based filtering • Determine item similarity based on item content not usage data • Recommend items similar to those that a user is known to like • The user model: • explicitly provided features/keywords of interest • can be a classifier (e.g Naive Bayes, SVM, Decision trees) Formal representation • ti: rating of user xi for item yi, where xi and yi are feature vectors • Infer prediction function

26. Content-based filtering • Determine item similarity based on item content not usage data • Recommend items similar to those that a user is known to like • The user model: • explicitly provided features/keywords of interest • can be a classifier (e.g Naive Bayes, SVM, Decision trees) Pros • No cold start problem • No need for usage data • No popularity bias, can recommend items with rare features Cons • Item content needs to be machine readable and meaningful • Easy to pigeonhole the user • Difficult to implement serendipity • Difficult to combine multiple item’ s features together

27. Hybrid approaches Method Description Weighted Outputs from several techniques (in the form of scores or votes) are combined with different degrees of importance to offer final recommendations Switching Depending on situation, the system changes from one technique to another Mixed Recommendations from several techniques are presented at the same time Feature combination Features from different recommendation sources are combined as input to a single technique Cascade The output from one technique is used as input of another that refines the result Feature augmentation The output from one technique is used as input features to another Meta-level The model learned by one recommender is used as input to another

28. Hybrid approaches Combining user and item features and usage to benefit from both Pros • Often outperforms CF and CB alone Cons • Can be a lot of work to get the right balance

29. Non-traditional approaches • Deep learning • Social recommendations • Learning to rank • ... Pros Cons • Good for eking out those final performance percentage points • You can say you’re working with current edge approaches ;) • Less well understood • Less supported in recommendation toolkits • Not recommended approaches for your first recommender

30. Algorithms • Typically employ collaborative filtering • May need to use content-based filtering particularly to bootstrap • Go advanced with a hybrid • Do all of that before getting adventurous with state-of-the-art Is your recommender doing well? You don’t really know unless you evaluate it...

32. Evaluating a recommender • Offline testing • Online testing (A/B testing)

33. Offline testing • Test offline before deploying • Parameter sweep is quick • Doesn’t offend real users • n-fold cross validation: • Take the users, items and relationships between them (e.g. clicked on, bought) • Split into n folds, for training (n-1) and testing (1) • Attempt to predict the testing data based on the training data • Popularity as baseline Metrics • Precision, recall and f-measure • Receiver operating characteristic (ROC) curve • Normalised discounted cumulative gain (NDCG) • Mean reciprocal rank (MRR) • Fraction of Concordant Pairs (FCP) • ...

34. Online testing • Offline performance isn’t a very precise indicator • Offline test is good sanity check • Online test gives real performance • A/B testing • Deploy your systems that perform ‘well enough’ • Compare them with each other in real world • Mind the pitfalls Metrics • The offline metrics + • Conversion rate • Open, view, click through rates • Usage data (e.g. reordered item, completed reading book) • Hard to evaluate: trustworthiness; freshness; diversity; serendipity; novelty; recency.

35. Evaluating a recommender • Start with offline testing • Perform A/B testing but be aware of the common pitfalls • Hard to evaluate performance in terms of: trustworthiness; freshness; diversity; serendipity; novelty; recency. How do we use recommenders? On to a few of our use cases...

37. Recommenders @ Mendeley

38. Recommenders @ Mendeley Related research for an article

39. Recommenders @ Mendeley Related research for multiple articles

40. Recommenders @ Mendeley Mendeley Suggest - personalised batch of recommended reading

41. Recommenders @ Mendeley Researchers to follow on Mendeley

42. Recommenders @ Mendeley Interesting activity from your social network

43. Recommenders @ Mendeley • Recommenders are employed for a number of use cases • Recommenders deliver different kinds of value depending upon use case • Can reuse the same underlying recommender system and framework for all

44. Conclusions • Recommenders are complementary to search and becoming mainstream • although arguably can cater for a wider range of use cases • When building a recommender, it’s common to predict ratings, rerank, optimise the page and then introduce context-awareness • In building a recommender, start with collaborative filtering if you can, content-based if you need to bootstrap and then explore hybrids • Open research questions remain as recommenders are used to tackle trustworthiness; freshness; diversity; serendipity; novelty; recency

45. References • Xavier Amatriain, The Recommender Problem Revisited (http://www. slideshare.net/xamat/recsys-2014-tutorial-the-recommender-problem-revisited) • Rec Sys 2014 (http://recsys.acm.org/recsys14/)

46. Thank you www.mendeley.com