Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Recommender systemms search engines


Published on

  • Be the first to comment

  • Be the first to like this

Recommender systemms search engines

  1. 1. RECOMMENDER SYSTEMS ANDSEARCH ENGINES – TWO SIDES OFTHE SAME COIN!?Bracha ShapiraLior RokachDepartment of Information Systems EngineeringBen-Gurion University
  2. 2. CONTENT Introduction Applications Methods Recommender Systems vs. search engines
  3. 3. ARE YOU BEING SERVED? What are you looking for? Demographic – Age, Gender, etc. Context-  Casual/Event  Season  Gift Purchase History  Loyal Customer  What is the customer currently wearing?  Style  Color Social  Friends and Family  Companion
  4. 4. RECOMMENDER SYSTEMS A recommender system (RS) helps people that have not sufficient personal experience or competence to evaluate the, potentially overwhelming, number of alternatives offered by a Web site.  In their simplest form RSs recommend to their users personalized and ranked lists of items  Provide consumers with information to help them decide which items to purchase
  7. 7. WHAT MOVIE SHOULD I WATCH? • The Internet Movie Database (IMDb) provides information about actors, films, television shows, television stars, video games and production crew personnel. • Owned by since 1998 • 796,328 titles and 2,127,371 people • More than 50M users per month.
  8. 8. abcd The Nextflix prize story In October 2006, Netflix announced it would give a $1 million to whoever created a movie- recommending algorithm 10% better than its own. Within two weeks, the DVD rental company had received 169 submissions, including three that were slightly superior to Cinematch, Netflixs recommendation software After a month, more than a thousand programs had been entered, and the top scorers were almost halfway to the goal But what started out looking simple suddenly got hard. The rate of improvement began to slow. The same three or four teams clogged the top of the leader-board. Progress was almost imperceptible, and people began to say a 10 percent improvement might not be possible. Three years later, on 21st of September 2009, Netflix announced the winner. 13.10.2012
  10. 10. WHERE SHOULD I SPEND MY VACATION? I would like to escape from this ugly an tedious work life and relax for two weeks in a sunny place. I am fed up with these crowded and noisy places … just the sand and the sea … and some “adventure”. I would like to bring my wife and my children on a holiday … it should not be to expensive. I prefer mountainous places… not too far from home. Children parks, easy paths and good cuisine are a must. I want to experience the contact with a completely different culture. I would like to be fascinated by the people and learn to look at my life in a totally different way.
  11. 11. Usage in the market/products RecommendationState-of-the-art solutions Examined Solutions Method Commonness Jinni Taste Kid Nanocrowd Clerkdogs Criticker IMDb Flixster Movielens Netflix Shazam Pandora LastFM YooChoose Think Analytics Itunes AmazonCollaborative Filtering v v v v v v v v v v v vContent-Based Techniques v v v v v v v v v v vKnowledge-Based Techniques v v v v v v vStereotype-Based Recommender Systems v v v v v v vOntologies and Semantic Web Technologies for v v vRecommender SystemsHybrid Techniques v v v v v v vEnsemble Techniques for Improving v futureRecommendationContext Dependent Recommender Systems v v v v v vConversational/Critiquing Recommender v vSystemsCommunity Based Recommender Systems and v v v v vRecommender Systems 2.0 13.10.2012
  13. 13. Collaborative FilteringDescription  The method of making automatic predictions (filtering) about the interests of a user by collecting taste information from many users (collaborating). The underlying assumption of CF approach is that 1 Collaborative Filtering those who agreed in the past tend to agree again in the future. Selected Techniques kNN - Nearest Neighbor  SVD – Matrix Factorization  Similarity Weights Optimization (SWO)  13.10.2012
  14. 14. COLLABORATIVE FILTERING abcdThe Idea Trying to predict the opinion the user will have on the different items and be able to recommend the “best” items to each user based on: the user’s previous likings and the opinions of other like minded users Negative Rating ? Positive Rating 13.10.2012
  15. 15. How collaborative filtering works?“People who liked this also liked…” abcd abcd How it works User-to-User  Recommendations are made by finding users with similar tastes. Jane and Tim both liked Item 2 and disliked Item 3; it seems they might have similar taste, which suggests that in general Jane agrees with Tim. This makes Item 1 a good recommendation for Tim. This approach does not scale well for millions of Item users. to Item Item-to-Item  Recommendations are made by finding items that have similar appeal to many users. Tom and Sandra are two users who liked both Item 1 and Item 4. That suggests that, in general, people who User to liked Item 4 will also like item 1, so Item 1 will be recommended to Tim. This approach is scalable to User millions of users and millions of items. 13.10.2012
  16. 16. KNN - NEAREST NEIGHBOR Current User Users 1 1st item rate0 Dislike ? 1 01 Like abcd Prediction abcd Unknown Rating abcd Other Users 1 This user did not   There are other Items? 1  The prediction Unknown rate the item. We was made based users who rated 0 will try to predict a on the nearest the same item. rating according We are interested 1 neighbor. to his neighbors. in the Nearest abcdHamming Distance 1  The Hamming distance Neighbors. is named after Richard Hamming. 0 User Model = 1  In information theory, the Hamming distance between two strings of interaction abcd Nearest Neighbors equal length is the number of  We are looking 1 positions at which the corresponding abcd history the Nearest 1 symbols are different. for Neighbor. The Nearest  one with the 1 Neighbor lowest Hamming 0 14th item rate distance. Hamming 5 6 6 5 4 8 distance 13.10.2012
  17. 17. IMPORTANT ISSUES Cold Start Implicit/Explicit Rating Sparsity  Long Tail problem - many items in the Long Tail have only few ratings Portfolio Effect: Non Diversity Problem  It is not useful to recommend all movies by Antonio Banderas to a user who liked one of them in the past Beyond Popularity  Gray sheep problem Iformation Security  Misuse  Privacy
  19. 19. CONTENT-BASED RECOMMENDATION In content-based recommendations the system tries to recommend items that matches the User Profile. The Profile is based on items user has liked in the past or explicit interests that he defines. A content-based recommender system matches the profile of the item to the user profile to decide on its relevancy to the user.
  20. 20. SIMPLE EXAMPLE Read update User ProfileNew books Match User Profile Recommender Systems recommendation
  22. 22. Context-Based Recommender Systems abcd Overview  The recommender system uses additional data about the context of an item consumption.  For example, in the case of a restaurant the time or the location may be used to improve the recommendation compared to what could be performed without this additional source of information.  A restaurant recommendation for a Saturday evening when you go with your spouse should be different than a restaurant recommendation on a workday afternoon when you go with co-workers 13.10.2012
  23. 23. Context-Based Recommender Systems Motivating Examples  Recommend a vacation  Winter vs. summer  Recommend a purchase (e-retailer)  Gift vs. for yourself  Recommend a movie  To a student who wants to watch it on Saturday night with his girlfriend in a movie theater. 13.10.2012
  24. 24. Context-Based Recommender Systems Motivating Examples  Recommend music  The music that we like to hear is greatly affected by a context, such that can be thought of a mixture of our feelings (mood) and the situation or location (the theme) we associate it with.  Listen to Bruce Springteen "Born in USA" while driving along the 101.  Listening to Mozarts Magic Flute while walking in Salzburg. 13.10.2012
  25. 25. Information Discovery: Example“Tell me the music that I want to listen NOW" abcd abcd Details  An Interactive personalized WebRadio  A mood matrix propose a relationship between music and mood.  Ethnographic studies have shown that people choose music peaces according to their mood or mood change expectation. 13.10.2012
  26. 26. Context-Based Recommender Systems What simple recommendation techniques ignore?  What is the user when asking for a recommendation?  Where (and when) the user is ?  What does the user (e.g., improve his knowledge or really buy a product)?  Is the user or with other ?  Are there products to choose or only ? 13.10.2012
  27. 27. Context-Based Recommender Systems What simple recommendation techniques ignore?  What is the user when asking for a recommendation?  Where (and when) the user is ?  What does the user (e.g., improve his knowledge or really buy a product)?  Is the user or with other ?  Are there products to choose or only ? Plain recommendation technologies forget to take into account the user context. 13.10.2012
  28. 28. Context-Based Recommender Systems abcd Major obstacle for contextual computing  Obtain sufficient and reliable data describing the user context  Selecting the right information, i.e., relevant in a particular personalization task  Understand the impact of contextual dimensions on the personalization process  Computational model the contextual dimension in a more classical recommendation technology  For instance: how to extend Collaborative Filtering to include contextual dimensions? 13.10.2012
  29. 29. Context-Based Recommender Systems abcd Item Split - Intuition and Approach  Each item in the data base ( ) is a candidate for splitting  Context defines ( ) all possible splits of an item ratings vector  We test all the possible splits – we do not have many contextual features  We choose one split (using a single contextual feature) that maximizes an impurity measure and whose impurity is higher than a threshold 13.10.2012
  31. 31. Social Based (Trust based) Recommender Systems abcd Overview  Intuition – Users tend to receive advice from people they trust, i.e., from their friends.  Trusted friends can be defined explicitly by the users or inferred from social networks they are registered to. . 13.10.2012
  32. 32. TRUST- BASED COLLABORATIVE FILTERING Active users’ trusted friends Active user 3 ? Rating prediction
  33. 33. TRUST METRICS  Global metrics: computes a single global trust value for every single user (reputation on the network) b  Pros: d a  Based on the whole community opinion c  Cons:  Trust is subjective (controversial users)
  34. 34. TRUST METRICS (CONT.) Local metrics: predicts (different) trust scores that are personalized from the point of view of every single user Pros:  More accurate  Attack resistance Cons:  Ignoring the “wisdom of the crowd” b a d c
  36. 36. SEARCH ENGINES VS. RECOMMENDER SYSTEMS – Search Engines Recommender Systems Goal – answer users ad  Goal – recommend services hoc queries or items to user Input – user ad-hoc need  Input - user preferences defined as a query defined as a profile Output- ranked items relevant to user need  Output - ranked items based (based on her on her preferences preferences???) Methods - Mainly IR  Methods – variety of based methods methods, IR, ML, UM
  37. 37. NEW TRENDS …  “Understand” the user actual needs from her context  Personalize results according to the user preferences Search engines may use some recommender systems methods to achieve these goals
  38. 38. SEARCH ENGINES PERSONALIZATION METHODSADOPTED FROM RECOMMENDER SYSTEMS Collaborative filtering  User-based - Cross domain collaborative filtering is required??? Content-based  Search history Collaborative content-based  Collaborate on similar queries Context-based  Little research – difficult to evaluate  Locality, language, calendar Social-based  Friends I trust relating to the query domain  Notion of trust, expertise