Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Recommendations as a Conversation with the User

22,284 views

Published on

These slides are from a tutorial at the 5th ACM International Conference on Recommender Systems (RecSys 2011).

Recommender systems aim to provide users with products or content that satisfy the users' stated or inferred needs. The primary evaluation measures for recommender systems emphasize either the perceived relevance of the recommendations or the actions associated with those recommendations (e.g., purchases or clicks). Unfortunately, this transactional emphasis neglects how users interact with recommendations in the context of information seeking tasks. The effectiveness of this interaction determines the user's experience beyond a single transaction. This tutorial explores the role of recommendations as part of a conversation between the user and an information seeking system. The tutorial does not require any special background in interfaces or usability, and will focus on practical techniques to make recommender systems most effective for users.

Published in: Technology, Business

Recommendations as a Conversation with the User

  1. Recommendations as aConversation with the UserDaniel TunkelangPrincipal Data Scientist at LinkedIn Recruiting Solutions 1
  2. Introductions 2
  3. Let’s talk about how we talk with machines… 3
  4. Clifford Nass’s secret:1) Find a conclusion by a social science researcher.2) Change “People do X when interacting with other people.” to “People do X when interacting with a computer.”3) Profit! 4
  5. Let’s work on our relationship. 5
  6. Core MessageRecommendations are a conversation with the user.1) Consider asking vs. guessing.2) Ask good questions.3) Its ok to make mistakes… if you have a good explanation and adapt to feedback. 6
  7. Our goal: http://www.wilsoninfo.com/computerclipart.shtm l 7
  8. Overview1) Theory2) Examples3) Action Items 8
  9. 1) Theory 9
  10. Pragmatics: the Study of Conversation Paul Grice 10
  11. Grice’s Maxims of ConversationMaxim 1: QualityMaxim 2: QuantityMaxim 3: RelationMaxim 4: MannerH. P. Grice, "Logic and conversation” [1975] 11
  12. Maxim 1: Quality 12
  13. Quality: Above All, the Truth Xiao, Bo and Benbasat, Izak. 2011. "Product-Related Deception in E-Commerce: A Theoretical Perspective," MIS Quarterly, (35: 1) pp.169-195. 13
  14. Don’t Lie1) Don’t use “recommended” when you really mean “sponsored” or “excess inventory”.2) Optimize for the user’s utility.3) Apply a standard of evidence (quality, quantity) that you believe in. 14
  15. Maxim 1: Quantity 15
  16. Right Amount of Information1) Exchange small units of information.2) If recommendations supplement other content, consider overall cognitive load.3) Provide short, meaningful explanations. 16
  17. Maxim 3: Relation 17
  18. Relevant to the User1) Offer value to the user.2) Respect task context.3) Don’t be obnoxious. 18
  19. Maxim 4: Manner 19
  20. Relevant to the User1) Eschew obfuscation.2) Avoid ambiguity.3) Be brief.4) Be orderly. 20
  21. Another Perspective Gary Marchionini 21
  22. Human-Computer Information Retrieval Empower people to explore large-scale information but demand that people also take responsibility for this control by expending cognitive and physical energy.Marchionini, G., “Toward Human-Computer Information Retrieval” [2006] 22
  23. Principles of HCIR1) Do more than deliver relevant information: facilitate sensemaking.2) Increase user responsibility and control: require and reward effort.3) Adapt to increasingly knowledgeable users over time.4) Be engaging and fun to use! 23
  24. Facilitate Sensemaking 24
  25. Require and Reward Effort http://www.posterenvy.com/catalog/ask_why.jpg 25
  26. Adapt to User Knowledge 26
  27. Be Engaging! http://bluenile.com/ 27
  28. Applying the theory to…1) Personalized Recommendations2) Social Recommendations3) Item Recommendations 28
  29. Personalized Recommendations1) Be transparent about model so users gain insight.2) Allow users to modify models to correct mistakes.3) Solicit just enough information to provide value. 29
  30. Social Recommendations1) Identify the right set of similar users.2) Allow users to manipulate the social lens.3) Accommodate users who break your model. 30
  31. Item Recommendations1) Explain recommendations to users.2) Watch out for non-sequiturs (e.g., diapers -> beer).3) Play well with user-controlled filtering and sorting. 31
  32. 2) Examples 32
  33. 33
  34. Initial User Experience 34
  35. “It just takes 2 minutes…” 35
  36. Asking Before Guessing 36
  37. Let’s try some answers: 37
  38. Uh oh… 38
  39. Expressing my gustibus… 39
  40. New Star Trek = Yes; New Star Wars = No 40
  41. Testing my patience… 41
  42. Bring on the quality! 42
  43. And continue the conversation. 43
  44. Learning from Netflix1) Ask the user for help up front. But not too much help.2) Pay attention to what the user tells you!3) Give users value early and often.75% of Netflix views result from recommendations 44
  45. 45
  46. Initial User Experience 46
  47. Seed with an artist… 47
  48. Or track or genre. 48
  49. Goo Goo Gjoob! 49
  50. Ease user into recommendation space… 50
  51. And go wild! 51
  52. Shared Product: Personalized Stream 52
  53. Positive and Negative Feedback 53
  54. Learning from Pandora1) Get meaningful input from user in one step.2) Explain recommendations to users.3) Solicit feedback and act on it immediately. 54
  55. 55
  56. My home page… 56
  57. Explanations and Humility 57
  58. Explain What and Why 58
  59. Recommendations as a Starting Point 59
  60. Learning from Amazon1) Show the factors that drive your conclusions.2) Distinguish different kinds of recommendations.3) Combine recommendations with user control.Amazon: 35% of sales result from recommendations 60
  61. 3) Action Items 61
  62. Increase explainability.Explanations can be even more important than therecommendations themselves.Herlocker et al., “Explaining collaborative filtering recommendations” [2000]Sinha and Swearingen, “The role of transparency in recommender systems”[2002]Tintarev and Masthoff, “Effective explanations of recommendations: User-centered design” [2007](via Òscar Celma’s book, Music Recommendation and Discovery: The LongTail, Long Fail, and Long Play in the Digital Music Space) 62
  63. Some models more explainable than others.1) Consider decision trees and rule-based systems.2) Avoid using latent, unlabeled features.3) If the model is opaque, use examples as surrogates. 63
  64. Make a good first impression.Your user’s first experience is critical.Use popularity as a default if it makes sense.Solicit one valuable piece of information as quickly andpainlessly as possible.“Do you like the taste of beer?”http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/ 64
  65. Design feedback into your system.You can make mistakes, if users can easily fix them.Challenging if models use offline computation.Respond instantly; generalize as quickly as possible.Agarwal and Chen, “Machine Learning for Large Scale Recommender Systems”[ICML 2011 Tutorial] 65
  66. Integrate recommendations with search.Recommend next steps, not just items.In a task context, recommendations are just anothersource of information scent.Be careful in integrating offline recommendations withonline features like search and navigation.Pirolli, Information Foraging Theory: Adaptive Interaction with Information [2007] 66
  67. SummaryRecommendations are a conversation with the user.1) Consider asking vs. guessing.2) Ask good questions.3) Its ok to make mistakes… if you have a good explanation and adapt to feedback. 67
  68. Thank You! Questions? Contact: dtunkelang@linkedin.com We’re Hiring! http://engineering.linkedin.com/ 68

×