Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Search and Recommendation: 2 sides of the same coin?


Published on

Invited talk at the Workshop on the Future of Search organized in Andorra (April 2008) by Yahoo Research

Published in: Technology, Business

Search and Recommendation: 2 sides of the same coin?

  1. 1. TELEFÓNICA Research April 2008 Search and Recommendation: two sides of the same coin? a.k.a. Is the future of Search in Recommendation? © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Xavier Amatriain Researcher Where is the content?
  2. 2. <ul><li>01 Introduction </li></ul><ul><li>02 Recommender Systems </li></ul><ul><li>03 Search Engines </li></ul><ul><li>04 Are Search Engines going towards RS? </li></ul><ul><li>05 Conclusions </li></ul>Index
  3. 3. Information overload “People read around 10 MB worth of material a day, hear 400 MB a day, and see one MB of information every second” The Economist, November 2006
  4. 4. The Age of Search has come to an end <ul><li>... long live the Age of Recommendation! </li></ul><ul><li>In an article published in CNN Money, entitled “The race to create a 'smart' Google”, Fortune magazine writer Jeffrey M. O'Brien, writes: </li></ul><ul><ul><li>“The Web, they say, is leaving the era of search and entering one of discovery. What's the difference? Search is what you do when you're looking for something. Discovery is when something wonderful that you didn't know existed, or didn't know how to ask for, finds you.” </li></ul></ul><ul><ul><li>(Extracted from ACM Recsys 08 website) </li></ul></ul>
  5. 5. Recommender Systems in Telefonica Multimedia Entertainment E-commerce Social Networking News/Blogs/Portals Comunidades PLATFORM PRODUCTS AND SERVICES COMMERCIALIZATION Content Packaging and Design Devices Access Commercialization Customers Recommendation Systems
  6. 6. Access through PC for exploring the whole application features Browse, Search, Rate, Comment… Get recommended series Get recommended games Meet and interact with other people Use a little application for mobile devices Get recommendations What series I watch now? Browse across TuSerie with another devices and discover new user experiences Cross-platform recommendation systems
  7. 7. Social Networks and Recommendations Browse through user profile to check the compatibility with a specific user See the last events or the last interactions that your friends have made with the system Send messages and make this user a friend or discard him Check your compatibility with another user. View his ratings and compare them with yours. Search users
  8. 8. Domain-specific recommendation <ul><li>User profiling </li></ul><ul><ul><li>Implicit through log analysis </li></ul></ul><ul><ul><li>Explicit asking for user feedback </li></ul></ul><ul><li>Integrate search and recommendation in user-friendly interfaces </li></ul>Usuario Genérico User Profile Global Search Content Based Collaborative
  9. 9. Investment in Recommendation Companies
  10. 10. 02 Recommender Systems
  11. 11. The “Recommender problem” <ul><li>Estimate a utility function that is able to automatically predict how much a user will like an item that is unknown for her. Based on: </li></ul><ul><ul><li>Past behavior </li></ul></ul><ul><ul><li>Relations to other users </li></ul></ul><ul><ul><li>Item similarity </li></ul></ul><ul><ul><li>Context </li></ul></ul><ul><ul><li>... </li></ul></ul>Offline Online
  12. 12. Approaches to Recommendation <ul><li>Collaborative Filtering </li></ul><ul><ul><li>Recommend items based only on the users past behavior </li></ul></ul><ul><ul><ul><li>Similarity between users or items computed only from this </li></ul></ul></ul><ul><ul><li>User-based </li></ul></ul><ul><ul><ul><li>Find similar users to me and recommend what those users liked </li></ul></ul></ul><ul><ul><li>Item-based </li></ul></ul><ul><ul><ul><li>Find similar items to those that I have previously liked </li></ul></ul></ul><ul><li>Content-based </li></ul><ul><ul><li>Recommend based on features inherent to the items </li></ul></ul>
  13. 13. Recommendation as a Datamining problem <ul><li>The core of the Recommendation Engine has been assimilated to a general datamining problem: </li></ul><ul><li>However RS have attracted input from a large community </li></ul><ul><ul><li>IR, e-Commerce, HCI, Psychology... </li></ul></ul>
  14. 14. What works <ul><li>It depends on the domain: Domain-specific modeling </li></ul><ul><li>However, in the general case it has been demonstrated that (currently) the best isolated approach is CF. </li></ul><ul><ul><li>Item-based in general more efficient and better but mixing CF approaches can improve result </li></ul></ul><ul><ul><li>Other approaches can be hybridized to improve results in specific cases (cold-start problem...) </li></ul></ul><ul><li>What matters: </li></ul><ul><ul><li>Data preprocessing: outlier removal, denoising, removal of global effects (e.g. individual user's average) </li></ul></ul><ul><ul><li>“Smart” dimensionality reduction using MF such as SVD </li></ul></ul><ul><ul><li>Combining classifiers </li></ul></ul>
  15. 15. Data mining + all those other things <ul><li>User Interface </li></ul><ul><li>System requirements (efficiency, scalability, privacy....) </li></ul><ul><li>and .... </li></ul>
  16. 16. Serindipity <ul><li>Unsought finding </li></ul><ul><li>Don't recommend items the user already knows or would have found anyway . </li></ul><ul><li>Expand the user's taste into neighboring areas by improving the obvious </li></ul><ul><li>Collaborative filtering can offer controllable serendipity (e.g. controlling how many neighbors to use in the recommendation) </li></ul>
  17. 17. 02 Search
  18. 18. Search Engines <ul><li>General process diagram of a search engine </li></ul>Offline Online
  19. 19. Search vs. recommendation <ul><li>Is search a content-based “recommendation”? </li></ul><ul><ul><li>In the indexing and retrieval processes we are trying to “cluster” similar documents based exlusively on content (no user information) </li></ul></ul><ul><li>or a poor-man's approach to CF? </li></ul><ul><ul><li>Most ranking algorithms can be seen as a simplified collaborative filtering where we are recommended the opinion of the average user's (what most people link) or the authorities (e.g. Page Rank). </li></ul></ul><ul><ul><li>To some extent we can say that web “structure” reflects past users' behavior </li></ul></ul>
  20. 20. 03 But, is Search going towards Recommendation?
  21. 21. Personalized Search <ul><li>Last year's presentation in this same Workshop is a good starting point (Paul-Alexandru Chirita, Current Approaches to Personalize Web Search) </li></ul><ul><li>Overall trend -> Use personalized user profile in order to improve returned page ranking </li></ul>
  22. 22. Recent advances in Personalized Search <ul><li>Interesting approaches: </li></ul><ul><ul><li>Automatic Identification of User Interest for Personalized Search (Qiu et al WWW06) </li></ul></ul><ul><ul><ul><li>Improve topic-sensitive page rank by inferring topic preference vector for the user. </li></ul></ul></ul><ul><ul><ul><li>Very similar to content-based recommendation </li></ul></ul></ul><ul><ul><li>CubeSVD: A Novel Approach to Personalized Web search (Sun et al WWW05) </li></ul></ul><ul><ul><ul><li>LSI using HOSVD to find a score for webpages based on q,u pairs. </li></ul></ul></ul><ul><ul><ul><li>Very similar to CF </li></ul></ul></ul>
  23. 23. A hybrid Search-Recommender? == ? Recommendation Search
  24. 24. 04 Conclusions <ul><li>The ever-growing amount of content makes searching difficult (time-consuming and unsatisfactory) </li></ul><ul><ul><li>Too much to search for, too many results </li></ul></ul><ul><ul><li>Frustration from it not being adaptive </li></ul></ul><ul><li>Search is starting to take the user into account </li></ul><ul><li>Is search something users want to do or just something they can do with the tools we offer? </li></ul><ul><li>Are search and recommendation two sides of the same coin? </li></ul><ul><ul><li>Is search about retrieval and recommendation about ranking? </li></ul></ul><ul><ul><li>Should they complement each other or become the same thing? </li></ul></ul>
  25. 25. [email_address] http://www.tid.es We're hiring!