Recommendations as a Conversation with the User

19,685 views

Published on

These slides are from a tutorial at the 5th ACM International Conference on Recommender Systems (RecSys 2011).

Recommender systems aim to provide users with products or content that satisfy the users' stated or inferred needs. The primary evaluation measures for recommender systems emphasize either the perceived relevance of the recommendations or the actions associated with those recommendations (e.g., purchases or clicks). Unfortunately, this transactional emphasis neglects how users interact with recommendations in the context of information seeking tasks. The effectiveness of this interaction determines the user's experience beyond a single transaction. This tutorial explores the role of recommendations as part of a conversation between the user and an information seeking system. The tutorial does not require any special background in interfaces or usability, and will focus on practical techniques to make recommender systems most effective for users.

Published in: Technology, Business
1 Comment
32 Likes
Statistics
Notes
No Downloads
Views
Total views
19,685
On SlideShare
0
From Embeds
0
Number of Embeds
1,023
Actions
Shares
0
Downloads
258
Comments
1
Likes
32
Embeds 0
No embeds

No notes for slide
  • Herlockerer al: “Explanations provide us with a mechanism for handling errors that come with a recommendation…most users value the explanations and would like to see them added”Sinha and Swearingen: “Meanlikingwas significantly higher for transparent than non-transparent recommendations…Mean Confidence showed a similar trend”Tintarev and Mastoff: “Feature selection in explanations needs to be tailored to the user [and] context…Features can be selected from a relatively short list…Explanation source matters”
  • Recommendations as a Conversation with the User

    1. Recommendations as aConversation with the UserDaniel TunkelangPrincipal Data Scientist at LinkedIn Recruiting Solutions 1
    2. Introductions 2
    3. Let’s talk about how we talk with machines… 3
    4. Clifford Nass’s secret:1) Find a conclusion by a social science researcher.2) Change “People do X when interacting with other people.” to “People do X when interacting with a computer.”3) Profit! 4
    5. Let’s work on our relationship. 5
    6. Core MessageRecommendations are a conversation with the user.1) Consider asking vs. guessing.2) Ask good questions.3) Its ok to make mistakes… if you have a good explanation and adapt to feedback. 6
    7. Our goal: http://www.wilsoninfo.com/computerclipart.shtm l 7
    8. Overview1) Theory2) Examples3) Action Items 8
    9. 1) Theory 9
    10. Pragmatics: the Study of Conversation Paul Grice 10
    11. Grice’s Maxims of ConversationMaxim 1: QualityMaxim 2: QuantityMaxim 3: RelationMaxim 4: MannerH. P. Grice, "Logic and conversation” [1975] 11
    12. Maxim 1: Quality 12
    13. Quality: Above All, the Truth Xiao, Bo and Benbasat, Izak. 2011. "Product-Related Deception in E-Commerce: A Theoretical Perspective," MIS Quarterly, (35: 1) pp.169-195. 13
    14. Don’t Lie1) Don’t use “recommended” when you really mean “sponsored” or “excess inventory”.2) Optimize for the user’s utility.3) Apply a standard of evidence (quality, quantity) that you believe in. 14
    15. Maxim 1: Quantity 15
    16. Right Amount of Information1) Exchange small units of information.2) If recommendations supplement other content, consider overall cognitive load.3) Provide short, meaningful explanations. 16
    17. Maxim 3: Relation 17
    18. Relevant to the User1) Offer value to the user.2) Respect task context.3) Don’t be obnoxious. 18
    19. Maxim 4: Manner 19
    20. Relevant to the User1) Eschew obfuscation.2) Avoid ambiguity.3) Be brief.4) Be orderly. 20
    21. Another Perspective Gary Marchionini 21
    22. Human-Computer Information Retrieval Empower people to explore large-scale information but demand that people also take responsibility for this control by expending cognitive and physical energy.Marchionini, G., “Toward Human-Computer Information Retrieval” [2006] 22
    23. Principles of HCIR1) Do more than deliver relevant information: facilitate sensemaking.2) Increase user responsibility and control: require and reward effort.3) Adapt to increasingly knowledgeable users over time.4) Be engaging and fun to use! 23
    24. Facilitate Sensemaking 24
    25. Require and Reward Effort http://www.posterenvy.com/catalog/ask_why.jpg 25
    26. Adapt to User Knowledge 26
    27. Be Engaging! http://bluenile.com/ 27
    28. Applying the theory to…1) Personalized Recommendations2) Social Recommendations3) Item Recommendations 28
    29. Personalized Recommendations1) Be transparent about model so users gain insight.2) Allow users to modify models to correct mistakes.3) Solicit just enough information to provide value. 29
    30. Social Recommendations1) Identify the right set of similar users.2) Allow users to manipulate the social lens.3) Accommodate users who break your model. 30
    31. Item Recommendations1) Explain recommendations to users.2) Watch out for non-sequiturs (e.g., diapers -> beer).3) Play well with user-controlled filtering and sorting. 31
    32. 2) Examples 32
    33. 33
    34. Initial User Experience 34
    35. “It just takes 2 minutes…” 35
    36. Asking Before Guessing 36
    37. Let’s try some answers: 37
    38. Uh oh… 38
    39. Expressing my gustibus… 39
    40. New Star Trek = Yes; New Star Wars = No 40
    41. Testing my patience… 41
    42. Bring on the quality! 42
    43. And continue the conversation. 43
    44. Learning from Netflix1) Ask the user for help up front. But not too much help.2) Pay attention to what the user tells you!3) Give users value early and often.75% of Netflix views result from recommendations 44
    45. 45
    46. Initial User Experience 46
    47. Seed with an artist… 47
    48. Or track or genre. 48
    49. Goo Goo Gjoob! 49
    50. Ease user into recommendation space… 50
    51. And go wild! 51
    52. Shared Product: Personalized Stream 52
    53. Positive and Negative Feedback 53
    54. Learning from Pandora1) Get meaningful input from user in one step.2) Explain recommendations to users.3) Solicit feedback and act on it immediately. 54
    55. 55
    56. My home page… 56
    57. Explanations and Humility 57
    58. Explain What and Why 58
    59. Recommendations as a Starting Point 59
    60. Learning from Amazon1) Show the factors that drive your conclusions.2) Distinguish different kinds of recommendations.3) Combine recommendations with user control.Amazon: 35% of sales result from recommendations 60
    61. 3) Action Items 61
    62. Increase explainability.Explanations can be even more important than therecommendations themselves.Herlocker et al., “Explaining collaborative filtering recommendations” [2000]Sinha and Swearingen, “The role of transparency in recommender systems”[2002]Tintarev and Masthoff, “Effective explanations of recommendations: User-centered design” [2007](via Òscar Celma’s book, Music Recommendation and Discovery: The LongTail, Long Fail, and Long Play in the Digital Music Space) 62
    63. Some models more explainable than others.1) Consider decision trees and rule-based systems.2) Avoid using latent, unlabeled features.3) If the model is opaque, use examples as surrogates. 63
    64. Make a good first impression.Your user’s first experience is critical.Use popularity as a default if it makes sense.Solicit one valuable piece of information as quickly andpainlessly as possible.“Do you like the taste of beer?”http://blog.okcupid.com/index.php/the-best-questions-for-first-dates/ 64
    65. Design feedback into your system.You can make mistakes, if users can easily fix them.Challenging if models use offline computation.Respond instantly; generalize as quickly as possible.Agarwal and Chen, “Machine Learning for Large Scale Recommender Systems”[ICML 2011 Tutorial] 65
    66. Integrate recommendations with search.Recommend next steps, not just items.In a task context, recommendations are just anothersource of information scent.Be careful in integrating offline recommendations withonline features like search and navigation.Pirolli, Information Foraging Theory: Adaptive Interaction with Information [2007] 66
    67. SummaryRecommendations are a conversation with the user.1) Consider asking vs. guessing.2) Ask good questions.3) Its ok to make mistakes… if you have a good explanation and adapt to feedback. 67
    68. Thank You! Questions? Contact: dtunkelang@linkedin.com We’re Hiring! http://engineering.linkedin.com/ 68

    ×